Pickle incompatibility of numpy arrays between Python 2 and 3

by

This seems like some sort of incompatibility. It’s trying to load a “binstring” object, which is assumed to be ASCII, while in this case it is binary data. If this is a bug in the Python 3 unpickler, or a “misuse” of the pickler by numpy, I don’t know.

Here is something of a workaround, but I don’t know how meaningful the data is at this point:

import pickle
import gzip
import numpy

with open('mnist.pkl', 'rb') as f:
    u = pickle._Unpickler(f)
    u.encoding = 'latin1'
    p = u.load()
    print(p)

Unpickling it in Python 2 and then repickling it is only going to create the same problem again, so you need to save it in another format.

More Related Contents:

  1. Saving and loading multiple objects in pickle file?
  2. Using pickle.dump – TypeError: must be str, not bytes
  3. Unpickling a python 2 object with python 3
  4. Python 3 – Can pickle handle byte objects larger than 4GB?
  5. Python random.choice same output
  6. Python: read object from file in python
  7. Python Variable Declaration
  8. TypeError: ‘
  9. Local (?) variable referenced before assignment [duplicate]
  10. How to deal with certificates using Selenium?
  11. PyQt4 Wait in thread for user input from GUI
  12. If range() is a generator in Python 3.3, why can I not call next() on a range?
  13. Upload Image using POST form data in Python-requests
  14. Which classes cannot be subclassed?
  15. importing izip from itertools module gives NameError in Python 3.x
  16. Python 3.4.0 with MySQL database
  17. Why is the purpose of the “else” clause following a “for” or “while” loop? [duplicate]
  18. Regular expression parsing a binary file?
  19. problems with pygame controller support
  20. error installing nltk supporting packages : nltk.download()
  21. How can I add python to cmd in windows [closed]
  22. Extract a subset of key-value pairs from dictionary?
  23. Bare asterisk in function parameters?
  24. Pygame make sprite walk in given rotation
  25. How to reverse a dictionary (whose values are lists) in Python? [duplicate]
  26. Travel directory tree with limited recursion depth
  27. Sound generation / synthesis with python?
  28. Python 3 sqlite parameterized SQL-query
  29. How to compute jaccard similarity from a pandas dataframe
  30. Size of a Python list in memory

Leave a Comment