zlib.error: Error -3 while decompressing: incorrect header check

by

You have this error:

zlib.error: Error -3 while decompressing: incorrect header check

Which is most likely because you are trying to check headers that are not there, e.g. your data follows RFC 1951 (deflate compressed format) rather than RFC 1950 (zlib compressed format) or RFC 1952 (gzip compressed format).

choosing windowBits

But zlib can decompress all those formats:

  • to (de-)compress deflate format, use wbits = -zlib.MAX_WBITS
  • to (de-)compress zlib format, use wbits = zlib.MAX_WBITS
  • to (de-)compress gzip format, use wbits = zlib.MAX_WBITS | 16

See documentation in http://www.zlib.net/manual.html#Advanced (section inflateInit2)

examples

test data:

>>> deflate_compress = zlib.compressobj(9, zlib.DEFLATED, -zlib.MAX_WBITS)
>>> zlib_compress = zlib.compressobj(9, zlib.DEFLATED, zlib.MAX_WBITS)
>>> gzip_compress = zlib.compressobj(9, zlib.DEFLATED, zlib.MAX_WBITS | 16)
>>> 
>>> text=""'test'''
>>> deflate_data = deflate_compress.compress(text) + deflate_compress.flush()
>>> zlib_data = zlib_compress.compress(text) + zlib_compress.flush()
>>> gzip_data = gzip_compress.compress(text) + gzip_compress.flush()
>>>

obvious test for zlib:

>>> zlib.decompress(zlib_data)
'test'

test for deflate:

>>> zlib.decompress(deflate_data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
zlib.error: Error -3 while decompressing data: incorrect header check
>>> zlib.decompress(deflate_data, -zlib.MAX_WBITS)
'test'

test for gzip:

>>> zlib.decompress(gzip_data)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
zlib.error: Error -3 while decompressing data: incorrect header check
>>> zlib.decompress(gzip_data, zlib.MAX_WBITS|16)
'test'

the data is also compatible with gzip module:

>>> import gzip
>>> import StringIO
>>> fio = StringIO.StringIO(gzip_data)  # io.BytesIO for Python 3
>>> f = gzip.GzipFile(fileobj=fio)
>>> f.read()
'test'
>>> f.close()

automatic header detection (zlib or gzip)

adding 32 to windowBits will trigger header detection

>>> zlib.decompress(gzip_data, zlib.MAX_WBITS|32)
'test'
>>> zlib.decompress(zlib_data, zlib.MAX_WBITS|32)
'test'

using gzip instead

or you can ignore zlib and use gzip module directly; but please remember that under the hood, gzip uses zlib.

fh = gzip.open('abc.gz', 'rb')
cdata = fh.read()
fh.close()

More Related Contents:

  1. Python decompressing gzip chunk-by-chunk
  2. Unzipping part of a .gz file using python
  3. TypeError: ‘str’ does not support the buffer interface
  4. Does python urllib2 automatically uncompress gzip data fetched from webpage?
  5. no module named zlib
  6. Python decompression relative performance?
  7. How to unzip gz file using Python
  8. Download and decompress gzipped file in memory?
  9. Installing Python-2.7 on Ubuntu 10.4
  10. Parsing compressed xml feed into ElementTree
  11. How do I gzip compress a string in Python?
  12. sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings
  13. Get uncompressed size of a .gz file in python
  14. Short description of the scoping rules?
  15. Handling very large numbers in Python
  16. Why do you need explicitly have the “self” argument in a Python method?
  17. How do I get monitor resolution in Python?
  18. Python list multiplication: [[…]]*3 makes 3 lists which mirror each other when modified [duplicate]
  19. ImportError: No module named PIL
  20. Interactive input/output using Python
  21. How to find duplicate elements in array using for loop in Python?
  22. What’s the best way to get an HTTP response code from a URL?
  23. faster alternative to numpy.where?
  24. Networkx: Overlapping edges when visualizing MultiGraph
  25. Performance of zeros function in Numpy
  26. How can I render a tree structure (recursive) using a django template?
  27. Clear text from textarea with selenium
  28. How to specify credentials when connecting to boto3 S3?
  29. Clear valid form after it is submitted
  30. Python Append to a list returns none [duplicate]

Leave a Comment