Why can’t I iterate twice over the same data?

by

It’s because data is an iterator, and you can consume an iterator only once. For example:

lst = [1, 2, 3]
it = iter(lst)

next(it)
=> 1
next(it)
=> 2
next(it)
=> 3
next(it)
=> StopIteration

If we are traversing some data using a for loop, that last StopIteration will cause it to exit the first time. If we try to iterate over it again, we’ll keep getting the StopIteration exception, because the iterator has already been consumed.

Now for the second question: What if we do need to traverse the iterator more than once? A simple solution would be to save all the elements to a list, which can be traversed as many times as needed. For instance, if data is an iterator:

data = list(data)

That is alright as long as there are few elements in the list. However, if there are many elements, it’s a better idea to create independent iterators using tee():

import itertools
it1, it2 = itertools.tee(data, 2) # create as many as needed

Now we can loop over each one in turn:

for e in it1:
    print("doing this one time")

for e in it2:
    print("doing this two times")

More Related Contents:

  1. Modifying list while iterating [duplicate]
  2. Removing from a list while iterating over it
  3. Difference between Python’s Generators and Iterators
  4. How does zip(*[iter(s)]*n) work in Python?
  5. Iterate an iterator by chunks (of n) in Python?
  6. How can I iterate through two lists in parallel?
  7. zip variable empty after first use
  8. How to print a generator expression?
  9. What are iterator, iterable, and iteration?
  10. Why does defining __getitem__ on a class make it iterable in python?
  11. Can iterators be reset in Python?
  12. Update row values where certain condition is met in pandas
  13. Cleanest way to get last item from Python iterator
  14. If range() is a generator in Python 3.3, why can I not call next() on a range?
  15. Is close() necessary when using iterator on a Python file object [duplicate]
  16. Is there any built-in way to get the length of an iterable in python?
  17. Why does next raise a ‘StopIteration’, but ‘for’ do a normal return?
  18. has_next in Python iterators?
  19. Python range() and zip() object type
  20. How to iterate across lines in two files simultaneously?
  21. Fastest way to convert an iterator to a list
  22. How to read one single line of csv data in Python?
  23. How can I iterate twice over a given iterator (reuse the data, “reset” the iterator etc.)? Why does iterating twice give a blank result?
  24. How can I create a list of elements from an iterator (convert the iterator to a list)?
  25. What is the idiomatic way to iterate over a binary file?
  26. Best way to loop over a python string backwards
  27. Proper way to reset csv.reader for multiple iterations?
  28. Pass multiple parameters to concurrent.futures.Executor.map?
  29. How do I re-map python dict keys
  30. How can I count the number of items in an arbitrary iterable (such as a generator)?

Leave a Comment