Are list-comprehensions and functional functions faster than “for loops”?

by

The following are rough guidelines and educated guesses based on experience. You should timeit or profile your concrete use case to get hard numbers, and those numbers may occasionally disagree with the below.

A list comprehension is usually a tiny bit faster than the precisely equivalent for loop (that actually builds a list), most likely because it doesn’t have to look up the list and its append method on every iteration. However, a list comprehension still does a bytecode-level loop:

>>> dis.dis(<the code object for `[x for x in range(10)]`>)
 1           0 BUILD_LIST               0
             3 LOAD_FAST                0 (.0)
       >>    6 FOR_ITER                12 (to 21)
             9 STORE_FAST               1 (x)
            12 LOAD_FAST                1 (x)
            15 LIST_APPEND              2
            18 JUMP_ABSOLUTE            6
       >>   21 RETURN_VALUE

Using a list comprehension in place of a loop that doesn’t build a list, nonsensically accumulating a list of meaningless values and then throwing the list away, is often slower because of the overhead of creating and extending the list. List comprehensions aren’t magic that is inherently faster than a good old loop.

As for functional list processing functions: While these are written in C and probably outperform equivalent functions written in Python, they are not necessarily the fastest option. Some speed up is expected if the function is written in C too. But most cases using a lambda (or other Python function), the overhead of repeatedly setting up Python stack frames etc. eats up any savings. Simply doing the same work in-line, without function calls (e.g. a list comprehension instead of map or filter) is often slightly faster.

Suppose that in a game that I’m developing I need to draw complex and huge maps using for loops. This question would be definitely relevant, for if a list-comprehension, for example, is indeed faster, it would be a much better option in order to avoid lags (Despite the visual complexity of the code).

Chances are, if code like this isn’t already fast enough when written in good non-“optimized” Python, no amount of Python level micro optimization is going to make it fast enough and you should start thinking about dropping to C. While extensive micro optimizations can often speed up Python code considerably, there is a low (in absolute terms) limit to this. Moreover, even before you hit that ceiling, it becomes simply more cost efficient (15% speedup vs. 300% speed up with the same effort) to bite the bullet and write some C.

More Related Contents:

  1. List comprehension vs map
  2. What is the most efficient way to loop through dataframes with pandas?
  3. Nested For Loops Using List Comprehension
  4. How to frame two for loops in list comprehension python
  5. Why list comprehension is much faster than numpy for multiplying arrays?
  6. for or while loop to do something n times [duplicate]
  7. sum each value in a list of tuples
  8. Converting List Comprehensions to For Loops in Python
  9. Creating a Python list comprehension with an if and break
  10. How on earth does this code work
  11. Fastest way to check if a value exists in a list
  12. Why does concatenation of DataFrames get exponentially slower?
  13. Understanding the map function
  14. A faster strptime?
  15. Python string ‘join’ is faster (?) than ‘+’, but what’s wrong here?
  16. How does a Python for loop with iterable work?
  17. yield in list comprehensions and generator expressions
  18. Creating a list of dictionaries with same keys? [duplicate]
  19. why isn’t numpy.mean multithreaded?
  20. Why is Collections.counter so slow?
  21. list.extend and list comprehension [duplicate]
  22. Any way to speed up Python and Pygame?
  23. matplotlib savefig performance, saving multiple pngs within loop
  24. interleaving 2 lists of unequal lengths [duplicate]
  25. What is the meaning of list[:] in this code? [duplicate]
  26. Speed up writing to files
  27. Redo for loop iteration in Python
  28. improving python code in Monte Carlo simulation
  29. List comprehension scope error from Python debugger
  30. Square braces not required in list comprehensions when used in a function

Leave a Comment