Scrape web page data generated by javascript

by

You need to look at PhantomJS.

From their site:

PhantomJS is a headless WebKit with JavaScript API. It has fast and
native support for various web standards: DOM handling, CSS selector,
JSON, Canvas, and SVG.

Using the API you can script the “browser” to interact with that page and scrape the data you need. You can then do whatever you need with it; including passing it to a PHP script if necessary.

That being said, if at all possible try not to “scrape” the data. If there is an ajax call the page is making, maybe there is an API you can use instead? If not, maybe you can convince them to make one. That would of course be much easier and more maintainable than screen scraping.

More Related Contents:

  1. How to hide css and js version details from weppalyzer in PHP
  2. Convert JSON to array in Javascript
  3. Preventing my form to refresh using ajax
  4. Why is set.timeout not working in this php while loop?
  5. Download the text over image using jquery
  6. Tally value in dynamically generated textbox
  7. how can i extract text and image in PDF file using php or javascript [closed]
  8. Dynamic page load can’t load Java with page
  9. Javascript alert and back with php
  10. Parse an HTML string with JS
  11. “Mixed content blocked” when running an HTTP AJAX operation in an HTTPS page
  12. AngularJS POST Fails: Response for preflight has invalid HTTP status code 404
  13. Double quotes within php script echo
  14. Encrypt with PHP, Decrypt with Javascript (cryptojs)
  15. How to refresh page on back button click?
  16. Send FormData and String Data Together Through JQuery AJAX?
  17. How to prevent page from reloading after form submit – JQuery
  18. Removing # from url in Angularjs while having .run in routes
  19. how to assign javascript variable value to php variable
  20. Is it possible to get data from HTML forms into android while using webView?
  21. Encrypt string in PHP and decrypt in Node.js
  22. Putting default value in
  23. search data from html input in mysql
  24. Parse the JavaScript returned from BeautifulSoup
  25. Get the content (text) of an URL after Javascript has run with PHP
  26. Include CSS,javascript file in Yii Framework
  27. IE10/11 Ajax XHR error – SCRIPT7002: XMLHttpRequest: Network Error 0x2ef3
  28. How do I refresh a DIV content?
  29. Angularjs – simple form submit
  30. How to submit checkboxes from all pages with jQuery DataTables

Leave a Comment