Can iterators be reset in Python?

by

I see many answers suggesting itertools.tee, but that’s ignoring one crucial warning in the docs for it:

This itertool may require significant
auxiliary storage (depending on how
much temporary data needs to be
stored). In general, if one iterator
uses most or all of the data before
another iterator starts, it is faster
to use list() instead of tee().

Basically, tee is designed for those situation where two (or more) clones of one iterator, while “getting out of sync” with each other, don’t do so by much — rather, they say in the same “vicinity” (a few items behind or ahead of each other). Not suitable for the OP’s problem of “redo from the start”.

L = list(DictReader(...)) on the other hand is perfectly suitable, as long as the list of dicts can fit comfortably in memory. A new “iterator from the start” (very lightweight and low-overhead) can be made at any time with iter(L), and used in part or in whole without affecting new or existing ones; other access patterns are also easily available.

As several answers rightly remarked, in the specific case of csv you can also .seek(0) the underlying file object (a rather special case). I’m not sure that’s documented and guaranteed, though it does currently work; it would probably be worth considering only for truly huge csv files, in which the list I recommmend as the general approach would have too large a memory footprint.

More Related Contents:

  1. Difference between Python’s Generators and Iterators
  2. How to print a generator expression?
  3. What’s the shortest way to count the number of items in a generator/iterator?
  4. If range() is a generator in Python 3.3, why can I not call next() on a range?
  5. Is there an expression for an infinite iterator?
  6. When is not a good time to use python generators?
  7. Why does next raise a ‘StopIteration’, but ‘for’ do a normal return?
  8. How can I count the number of items in an arbitrary iterable (such as a generator)?
  9. Understanding generators in Python
  10. Generator expressions vs. list comprehensions
  11. Lazy Method for Reading Big File in Python?
  12. How to build a basic iterator?
  13. How can I iterate over files in a given directory?
  14. Get the first item from an iterable that matches a condition
  15. Iterate over all pairs of consecutive items in a list [duplicate]
  16. Circular list iterator in Python
  17. Resetting generator object in Python
  18. how to split an iterable in constant-size chunks
  19. How to len(generator()) [duplicate]
  20. Is there any built-in way to get the length of an iterable in python?
  21. Python generator that groups another iterable into groups of N
  22. What does “three dots” in Python mean when indexing what looks like a number?
  23. Cartesian product of a dictionary of lists
  24. Length of generator output [duplicate]
  25. How to convert an iterable to a stream?
  26. Making a python iterator go backwards?
  27. split a generator/iterable every n items in python (splitEvery)
  28. How to get the n next values of a generator in a list (python)
  29. Iterate over the lines of a string
  30. Performance Advantages to Iterators?

Leave a Comment