Saving and loading multiple objects in pickle file?

by

Two additions to Tim Peters’ accepted answer.

First, you need not store the number of items you pickled separately if you stop loading when you hit the end of the file:

def loadall(filename):
    with open(filename, "rb") as f:
        while True:
            try:
                yield pickle.load(f)
            except EOFError:
                break

items = loadall(myfilename)

This assumes the file contains only pickles; if there’s anything else in there, the generator will try to treat whatever else is in there as pickles too, which could be dangerous.

Second, this way, you do not get a list but rather a generator.
This will load only one item into memory at a time, which is useful
if the dumped data is very large — one possible reason why you may
have wanted to pickle multiple items separately in the first place.
You can still iterate over items with a for loop as if it were
a list.

More Related Contents:

  1. Name class error
  2. How to print instances of a class using print()?
  3. How do I call a parent class’s method from a child class in Python?
  4. Why is the id of a Python class not unique when called quickly?
  5. Referring to the null object in Python
  6. Getting an instance name inside class __init__() [duplicate]
  7. What is the difference between __init__ and __call__?
  8. List attributes of an object [duplicate]
  9. Unpickling a python 2 object with python 3
  10. How do I detect if the mouse is hovering over a button? PyGame button class is not displaying the text or changing colour on hover
  11. Which classes cannot be subclassed?
  12. Prevent creating new attributes outside __init__
  13. Pickle incompatibility of numpy arrays between Python 2 and 3
  14. putting current class as return type annotation [duplicate]
  15. Why is `object` an instance of `type` and `type` an instance of `object`?
  16. How do I correctly setup and teardown for my pytest class with tests?
  17. Python 3 – Can pickle handle byte objects larger than 4GB?
  18. Why is a class __dict__ a mappingproxy?
  19. How to check if an object has an attribute in Python?
  20. How can I create an object and add attributes to it?
  21. Comparable classes in Python 3
  22. Class (Static) variables and methods in Python
  23. How to check if an object has an attribute?
  24. Using Properties in Python classes cause “maximum recursion depth exceeded” [duplicate]
  25. Saving and loading objects and using pickle
  26. “Enabling” comparison for classes [duplicate]
  27. How do you change the value of one attribute by changing the value of another? (dependent attributes)
  28. TypeError: attack() missing 1 required positional argument: ‘self’
  29. In Python, when are two objects the same?
  30. Provide __classcell__ example for Python 3.6 metaclass

Leave a Comment