How to save “complete webpage” not just basic html using Python

by

Try emulating your browser with selenium. This script will pop up the save as dialog for the webpage. You will still have to figure out how to emulate pressing enter for download to start as the file dialog is out of selenium’s reach (how you do it is also OS dependent).

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys

br = webdriver.Firefox()
br.get('http://www.google.com/')

save_me = ActionChains(br).key_down(Keys.CONTROL)\
         .key_down('s').key_up(Keys.CONTROL).key_up('s')
save_me.perform()

Also I think following @Amber suggestion of grabbing the the linked resources may be a simpler, thus a better solution. Still, I think using selenium is a good starting point as br.page_source will get you the entire dom along with the dynamic content generated by javascript.

More Related Contents:

  1. Python: URLError:
  2. Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str
  3. urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error
  4. What are the differences between the urllib, urllib2, urllib3 and requests module?
  5. Downloading a picture via urllib and python
  6. How to retrieve the values of dynamic html content using Python
  7. Python handling socket.error: [Errno 104] Connection reset by peer
  8. Python URLLib / URLLib2 POST
  9. python ignore certificate validation urllib2
  10. Login to website using urllib2 – Python 2.7
  11. Handling urllib2’s timeout? – Python
  12. Making a POST call instead of GET using urllib2
  13. What should I do if socket.setdefaulttimeout() is not working?
  14. How to download any(!) webpage with correct charset in python?
  15. Making HTTP HEAD request with urllib2 from Python 2
  16. How do I catch a specific HTTP error in Python?
  17. Submitting to a web form using python
  18. Why do I get “‘str’ object has no attribute ‘read'” when trying to use `json.load` on a string? [duplicate]
  19. How to resolve URLError:
  20. Python3 error: initial_value must be str or None, with StringIO
  21. Creating multiple python dictionaries from a file
  22. What is the result of % in Python?
  23. How to use __setattr__ correctly, avoiding infinite recursion
  24. How to get selectors with dynamic part inside using Selenium with Python?
  25. Passing a matplotlib figure to HTML (flask)
  26. ValueError: malformed string when using ast.literal_eval
  27. What does “table” in the string.translate function mean?
  28. no module named urllib.parse (How should I install it?)
  29. BeautifulSoup getText from between , not picking up subsequent paragraphs
  30. What is absolute import in python?

Leave a Comment