_pickle in python3 doesn’t work for large data saving

by

Not anymore in Python 3.4 which has PEP 3154 and Pickle 4.0
https://www.python.org/dev/peps/pep-3154/

But you need to say you want to use version 4 of the protocol:
https://docs.python.org/3/library/pickle.html

pickle.dump(d, open("file", 'w'), protocol=4)

More Related Contents:

  1. Saving an Object (Data persistence)
  2. Python multiprocessing PicklingError: Can’t pickle
  3. Saving and loading multiple objects in pickle file?
  4. How can I use pickle to save a dict (or any other Python object)?
  5. Multiprocessing: How to use Pool.map on a function defined in a class?
  6. Storing Python dictionaries
  7. Using pickle.dump – TypeError: must be str, not bytes
  8. Can Python pickle lambda functions?
  9. Unpickling a python 2 object with python 3
  10. Unable to load files using pickle and multiple modules
  11. How can I pickle a dynamically created nested class in python?
  12. Can’t pickle defaultdict
  13. Python pickling after changing a module’s directory
  14. How to read pickle file?
  15. Preserve custom attributes when pickling subclass of numpy array
  16. installing cPickle with python 3.5
  17. What causes the error “_pickle.UnpicklingError: invalid load key, ‘ ‘.”?
  18. What is faster – Loading a pickled dictionary object or Loading a JSON file – to a dictionary? [closed]
  19. multiprocessing.Pool – PicklingError: Can’t pickle : attribute lookup thread.lock failed
  20. Is pickle file of python cross-platform?
  21. Saving and loading objects and using pickle
  22. ImportError: No module named ‘pandas.indexes’
  23. What is the difference between rb and r+b modes in file objects [duplicate]
  24. pickle – putting more than 1 object in a file? [duplicate]
  25. pickle.load() raising EOFError in Windows
  26. Python Pickling Dictionary EOFError
  27. Common use-cases for pickle in Python
  28. Dump a list in a pickle file and retrieve it back later [closed]
  29. How to hash a large object (dataset) in Python?
  30. cProfile causes pickling error when running multiprocessing Python code

Leave a Comment