Performance Advantages to Iterators?

by

There’s actually a very good mail on the python mailing list about this: Iterators vs Lists. It’s a bit dated (from 2003), but as far as I know, it’s still valid.

Here’s the summary:

For small datasets, iterator and list based approaches have similar
performance.
For larger datasets, iterators save both time and space.

What I would draw from it is this: iterators are to be preferred over loading data into a list if possible. But unless you have a big dataset, don’t contort your code to make something that should fit in a list to work with an iterator.

More Related Contents:

  1. Is doing multiplication multiple times or assigning to a variable faster?
  2. How can I iterate over files in a given directory?
  3. Get the first item from an iterable that matches a condition
  4. pandas loc vs. iloc vs. at vs. iat?
  5. What is the best way to generate all possible three letter strings?
  6. Why is [] faster than list()?
  7. Why is numpy’s einsum faster than numpy’s built in functions?
  8. Speed comparison with Project Euler: C vs Python vs Erlang vs Haskell
  9. What’s a faster operation, re.match/search or str.find?
  10. Cost of exception handlers in Python
  11. File read using “open()” vs “with open()” [duplicate]
  12. It is more efficient to use if-return-return or if-else-return?
  13. Most efficient way to forward-fill NaN values in numpy array
  14. [] and {} vs list() and dict(), which is better?
  15. Numpy: Fix array with rows of different lengths by filling the empty elements with zeros
  16. How to convert an iterable to a stream?
  17. python: deque vs list performance comparison
  18. multiprocessing.Pool() slower than just using ordinary functions
  19. Efficiently Calculating a Euclidean Distance Matrix Using Numpy
  20. Complexity of list.index(x) in Python
  21. Python range() and zip() object type
  22. Why is pow(a, d, n) so much faster than a**d % n?
  23. Vectorizing or Speeding up Fuzzywuzzy String Matching on PANDAS Column
  24. What is the fastest way to output large DataFrame into a CSV file?
  25. How expensive are Python dictionaries to handle?
  26. Getting days since last occurence in Pandas DataFrame?
  27. Which is the most efficient way to iterate through a list in python?
  28. Is there a reason Python 3 enumerates slower than Python 2?
  29. Are any of these quad-tree libraries any good?
  30. Length of longest word in a list

Leave a Comment