What is the most efficient way to search nested lists in python?

by 
>>> lis=[['a','b','c'],['d','e','f']]
>>> any('d' in x for x in lis)
True

generator expression using any

$ python -m timeit -s "lis=[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]" "any('d' in x for x in lis)" 
1000000 loops, best of 3: 1.32 usec per loop

generator expression

$ python -m timeit -s "lis=[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]" "'d' in (y for x in lis for y in x)"
100000 loops, best of 3: 1.56 usec per loop

list comprehension

$ python -m timeit -s "lis=[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]" "'d' in [y for x in lis for y in x]"
100000 loops, best of 3: 3.23 usec per loop

How about if the item is near the end, or not present at all? any is faster than the list comprehension

$ python -m timeit -s "lis=[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]"
    "'NOT THERE' in [y for x in lis for y in x]"
100000 loops, best of 3: 4.4 usec per loop

$ python -m timeit -s "lis=[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]" 
    "any('NOT THERE' in x for x in lis)"
100000 loops, best of 3: 3.06 usec per loop

Perhaps if the list is 1000 times longer? any is still faster

$ python -m timeit -s "lis=1000*[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]"
    "'NOT THERE' in [y for x in lis for y in x]"
100 loops, best of 3: 3.74 msec per loop
$ python -m timeit -s "lis=1000*[['a','b','c'],['d','e','f'],[1,2,3],[4,5,6],[7,8,9],[10,11,12],[13,14,15],[16,17,18]]" 
    "any('NOT THERE' in x for x in lis)"
100 loops, best of 3: 2.48 msec per loop

We know that generators take a while to set up, so the best chance for the LC to win is a very short list

$ python -m timeit -s "lis=[['a','b','c']]"
    "any('c' in x for x in lis)"
1000000 loops, best of 3: 1.12 usec per loop
$ python -m timeit -s "lis=[['a','b','c']]"
    "'c' in [y for x in lis for y in x]"
1000000 loops, best of 3: 0.611 usec per loop

And any uses less memory too

More Related Contents:

  1. How to leave/exit/deactivate a Python virtualenv
  2. Correct way to try/except using Python requests module?
  3. Get all the diagonals in a matrix/list of lists in Python
  4. Splitting a list based on a delimiter word
  5. Intersecting two dictionaries
  6. Python convert tuple to string
  7. Is the shortcircuit behaviour of Python’s any/all explicit?
  8. How to ssh connect through Python Paramiko with ppk public key
  9. Iterating through two lists in Django templates
  10. Pythonic way to combine for-loop and if-statement
  11. Get data from pandas into a SQL server with PYODBC
  12. Kill a running subprocess call
  13. Iterating over a numpy array
  14. An integer is required? open()
  15. binning a dataframe in pandas in Python [duplicate]
  16. What is the difference between pandas agg and apply function?
  17. Efficient outer product in python
  18. Python wildcard search in string
  19. How can I add objects to a “pygame.sprite.Group()”?
  20. Python: Remove Duplicate Items from Nested list
  21. Is there a way to make the Tkinter text widget read only?
  22. Finding the full width half maximum of a peak
  23. Communication between two python scripts
  24. How to prevent airflow from backfilling dag runs?
  25. smartest way to join two lists into a formatted string
  26. Excel VLOOKUP equivalent in pandas
  27. Strange behaviour of the loss function in keras model, with pretrained convolutional base
  28. How to resolve the following error in discord.py: “TypeError __init__() missing 1 required keyword-only argument: ‘intents'” [duplicate]
  29. pandas scatter plotting datetime
  30. How can I achieve a self-referencing many-to-many relationship on the SQLAlchemy ORM back referencing to the same attribute?

Leave a Comment