Get size of a file before downloading in Python

by

I have reproduced what you are seeing:

import urllib, os
link = "http://python.org"
print "opening url:", link
site = urllib.urlopen(link)
meta = site.info()
print "Content-Length:", meta.getheaders("Content-Length")[0]

f = open("out.txt", "r")
print "File on disk:",len(f.read())
f.close()


f = open("out.txt", "w")
f.write(site.read())
site.close()
f.close()

f = open("out.txt", "r")
print "File on disk after download:",len(f.read())
f.close()

print "os.stat().st_size returns:", os.stat("out.txt").st_size

Outputs this:

opening url: http://python.org
Content-Length: 16535
File on disk: 16535
File on disk after download: 16535
os.stat().st_size returns: 16861

What am I doing wrong here? Is os.stat().st_size not returning the correct size?

Edit:
OK, I figured out what the problem was:

import urllib, os
link = "http://python.org"
print "opening url:", link
site = urllib.urlopen(link)
meta = site.info()
print "Content-Length:", meta.getheaders("Content-Length")[0]

f = open("out.txt", "rb")
print "File on disk:",len(f.read())
f.close()


f = open("out.txt", "wb")
f.write(site.read())
site.close()
f.close()

f = open("out.txt", "rb")
print "File on disk after download:",len(f.read())
f.close()

print "os.stat().st_size returns:", os.stat("out.txt").st_size

this outputs:

$ python test.py
opening url: http://python.org
Content-Length: 16535
File on disk: 16535
File on disk after download: 16535
os.stat().st_size returns: 16535

Make sure you are opening both files for binary read/write.

// open for binary write
open(filename, "wb")
// open for binary read
open(filename, "rb")

More Related Contents:

  1. How to download a file over HTTP?
  2. urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error
  3. UnicodeEncodeError: ‘charmap’ codec can’t encode characters
  4. How to urlencode a querystring in Python?
  5. urllib2.HTTPError: HTTP Error 403: Forbidden
  6. How to retrieve the values of dynamic html content using Python
  7. How can I percent-encode URL parameters in Python?
  8. Python: URLError:
  9. Python URLLib / URLLib2 POST
  10. Download Returned Zip file from URL
  11. Django: add image in an ImageField from image url
  12. How do I set headers using python’s urllib?
  13. How to handle response encoding from urllib.request.urlopen() , to avoid TypeError: can’t use a string pattern on a bytes-like object
  14. Python ‘requests’ library – define specific DNS?
  15. Handling urllib2’s timeout? – Python
  16. SSL: CERTIFICATE_VERIFY_FAILED with Python3
  17. How to unquote a urlencoded unicode string in python?
  18. Replace special characters in a string in Python
  19. How to know if urllib.urlretrieve succeeds?
  20. Making a POST call instead of GET using urllib2
  21. What should I do if socket.setdefaulttimeout() is not working?
  22. Changing User Agent in Python 3 for urrlib.request.urlopen
  23. Only add to a dict if a condition is met
  24. How do I catch a specific HTTP error in Python?
  25. Submitting to a web form using python
  26. no module named urllib.parse (How should I install it?)
  27. Download pdf using urllib?
  28. Python 3 – urllib, HTTP Error 407: Proxy Authentication Required
  29. Python3 error: initial_value must be str or None, with StringIO
  30. Python 3.6 urllib TypeError: can’t concat bytes to str

Leave a Comment