How to catch 404 error in urllib.urlretrieve

by

Check out urllib.urlretrieve‘s complete code:

def urlretrieve(url, filename=None, reporthook=None, data=None):
  global _urlopener
  if not _urlopener:
    _urlopener = FancyURLopener()
  return _urlopener.retrieve(url, filename, reporthook, data)

In other words, you can use urllib.FancyURLopener (it’s part of the public urllib API). You can override http_error_default to detect 404s:

class MyURLopener(urllib.FancyURLopener):
  def http_error_default(self, url, fp, errcode, errmsg, headers):
    # handle errors the way you'd like to

fn, h = MyURLopener().retrieve(url, reporthook=my_report_hook)

More Related Contents:

  1. How to download a file over HTTP?
  2. urllib2.HTTPError: HTTP Error 403: Forbidden
  3. How can I percent-encode URL parameters in Python?
  4. Download Returned Zip file from URL
  5. How do I set headers using python’s urllib?
  6. Python ‘requests’ library – define specific DNS?
  7. How do I send a POST request as a JSON?
  8. What should I do if socket.setdefaulttimeout() is not working?
  9. How do I catch a specific HTTP error in Python?
  10. Python – make a POST request using Python 3 urllib
  11. How do I set HTTP headers using Python’s urllib?
  12. What is a “slug” in Django?
  13. Retrieving parameters from a URL
  14. What is the quickest way to HTTP GET in Python?
  15. Using an HTTP PROXY – Python [duplicate]
  16. How to specify an authenticated proxy for a python http connection?
  17. Django: add image in an ImageField from image url
  18. Python-Requests close http connection
  19. Add params to given URL in Python
  20. Multithreaded web server in python
  21. Best way to parse a URL query string
  22. How to download a file using python in a ‘smarter’ way?
  23. How to get response SSL certificate from requests in python?
  24. Making a POST call instead of GET using urllib2
  25. multiprocessing.pool.MaybeEncodingError: ‘TypeError(“cannot serialize ‘_io.BufferedReader’ object”,)’
  26. Unescape Python Strings From HTTP
  27. Only add to a dict if a condition is met
  28. Submitting to a web form using python
  29. Download pdf using urllib?
  30. Python 3.6 urllib TypeError: can’t concat bytes to str

Leave a Comment