Is this possible to load the page after the javascript execute using python?

by

I ran into a similar problem when writing web scrapers in python, and I found Selenium Web Driver in combination with BeautifulSoup very useful. The code ends up looking something like this:

from selenium import webdriver
browser = webdriver.Firefox()
browser.get("http://www.yoursite.com")
soup = BeautifulSoup(browser.page_source, "html.parser")
...

With Selenium WebDriver, there’s also functionally for a “wait until a certain DOM element has loaded”, which makes the timing with javascript elements easier too.

More Related Contents:

  1. Parse the JavaScript returned from BeautifulSoup
  2. Can I scrape the raw data from highcharts.js?
  3. jQuery/JavaScript collision detection
  4. What browsers support HTML5 WebSocket API?
  5. Prevent jQuery UI dialog from setting focus to first textbox
  6. JavaScript: Is there a way to get Chrome to break on all errors?
  7. Detecting Unsaved Changes
  8. How to search JSON tree with jQuery
  9. angular ng-repeat in reverse
  10. “this” inside object [duplicate]
  11. Inject TypeORM repository into NestJS service for mock data testing
  12. Please point out the misfacts in my learning regarding Asynchronous Javascript
  13. Pure Javascript listen to input value change
  14. XMLHttpRequest module not defined/found
  15. Using anchor tags in react-router 4
  16. Vue JS: Difference of data() { return {} } vs data:() => ({ })
  17. How to replace url parameter with javascript/jquery?
  18. jQuery sort elements using data id
  19. Push Object in Array [duplicate]
  20. Why does getElementById not work on elements inside the document element? [duplicate]
  21. webgl readpixels is always returning 0,0,0,0
  22. Why is there no logical xor in JavaScript?
  23. How to use Array.prototype.filter with async?
  24. What is client side javascript and what is server side javascript?
  25. Prevent iOS bounce without disabling scroll ability
  26. How to generate unique ID with node.js
  27. knockout.js using $index with if binding
  28. Trying to use React.DOM to set body styles
  29. Error when requiring discordjs: “cannot find module ‘node:events'” [duplicate]
  30. Ruby on Rails – Send JavaScript variable from controller to external Javascript asset file

Leave a Comment