Common use-cases for pickle in Python

by

Some uses that I have come across:

1) saving a program’s state data to disk so that it can carry on where it left off when restarted (persistence)

2) sending python data over a TCP connection in a multi-core or distributed system (marshalling)

3) storing python objects in a database

4) converting an arbitrary python object to a string so that it can be used as a dictionary key (e.g. for caching & memoization).

There are some issues with the last one – two identical objects can be pickled and result in different strings – or even the same object pickled twice can have different representations. This is because the pickle can include reference count information.

To emphasise @lunaryorn’s comment – you should never unpickle a string from an untrusted source, since a carefully crafted pickle could execute arbitrary code on your system. For example see https://blog.nelhage.com/2011/03/exploiting-pickle/

More Related Contents:

  1. Saving an Object (Data persistence)
  2. Serializing class instance to JSON
  3. Python serialization – Why pickle?
  4. How to change the serialization method used by the multiprocessing module?
  5. Simple example of use of __setstate__ and __getstate__
  6. Making object JSON serializable with regular encoder
  7. Saving and loading multiple objects in pickle file?
  8. Multiprocessing: How to use Pool.map on a function defined in a class?
  9. Unable to load files using pickle and multiple modules
  10. How can I pickle a dynamically created nested class in python?
  11. Convert a python dict to a string and back
  12. Can’t pickle defaultdict
  13. django-rest-framework 3.0 create or update in nested serializer
  14. How do I serialize a Python dictionary into a string, and then back to a dictionary?
  15. Python pickling after changing a module’s directory
  16. How to read pickle file?
  17. Flask jsonify a list of objects
  18. What causes the error “_pickle.UnpicklingError: invalid load key, ‘ ‘.”?
  19. What is faster – Loading a pickled dictionary object or Loading a JSON file – to a dictionary? [closed]
  20. multiprocessing.Pool – PicklingError: Can’t pickle : attribute lookup thread.lock failed
  21. Is pickle file of python cross-platform?
  22. ImportError: No module named ‘pandas.indexes’
  23. What is the difference between rb and r+b modes in file objects [duplicate]
  24. pickle – putting more than 1 object in a file? [duplicate]
  25. Serialize the @property methods in a Python class
  26. Python Pickling Dictionary EOFError
  27. Dump a list in a pickle file and retrieve it back later [closed]
  28. How to hash a large object (dataset) in Python?
  29. cProfile causes pickling error when running multiprocessing Python code
  30. What difference between pickle and _pickle in python 3?

Leave a Comment