Why do list comprehensions write to the loop variable, but generators don’t? [duplicate]

by

Python’s creator, Guido van Rossum, mentions this when he wrote about generator expressions that were uniformly built into Python 3: (emphasis mine)

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension “leaks” the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original implementation of list comprehensions; it was one of Python’s “dirty little secrets” for years. It started out as an intentional compromise to make list comprehensions blindingly fast, and while it was not a common pitfall for beginners, it definitely stung people occasionally. For generator expressions we could not do this. Generator expressions are implemented using generators, whose execution requires a separate execution frame. Thus, generator expressions (especially if they iterate over a short sequence) were less efficient than list comprehensions.

However, in Python 3, we decided to fix the “dirty little secret” of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) 🙂 will print ‘before’, proving that the ‘x’ in the list comprehension temporarily shadows but does not override the ‘x’ in the surrounding scope.

So in Python 3 you won’t see this happen anymore.

Interestingly, dict comprehensions in Python 2 don’t do this either; this is mostly because dict comprehensions were backported from Python 3 and as such already had that fix in them.

There are some other questions that cover this topic too, but I’m sure you have already seen those when you searched for the topic, right? 😉

More Related Contents:

  1. Generator expressions vs. list comprehensions
  2. Why is a list comprehension so much faster than appending to a list?
  3. List vs generator comprehension speed with join function [duplicate]
  4. yield in list comprehensions and generator expressions
  5. What does the “yield” keyword do in Python?
  6. sum each value in a list of tuples
  7. flatten list of list through list comprehension
  8. range(len(list)) or enumerate(list)? [duplicate]
  9. Skip elements on a condition based in a list comprehension in python
  10. How does all() in python work on empty lists
  11. What is wrong with my python code? [duplicate]
  12. how to count average sentence length (in words) from a text file contains 100 sentences using python [closed]
  13. What needs to be added
  14. Return in generator together with yield in Python 3.3
  15. Split a generator into chunks without pre-walking it
  16. Launch a completely independent process
  17. Why does str(float) return more digits in Python 3 than Python 2?
  18. Add pygame module in PyCharm IDE
  19. ReactorNotRestartable error in while loop with scrapy
  20. How to read Unicode input and compare Unicode strings in Python?
  21. Absolute imports in python not working, relative imports work
  22. The truth value of an array with more than one element is ambigous when trying to index an array
  23. Networkx: Overlapping edges when visualizing MultiGraph
  24. How can I use list comprehensions to process a nested list?
  25. How to remove duplicate items from a list using list comprehension? [duplicate]
  26. Creating a Python list comprehension with an if and break
  27. Do python’s variable length arguments (*args) expand a generator at function call time?
  28. How to parallelize list-comprehension calculations in Python?
  29. Unknown error: Chrome failed to start: exited abnormally
  30. Coroutine vs Continuation vs Generator

Leave a Comment