Most efficient way for a lookup/search in a huge list (python)

by

Don’t create a list, create a set. It does lookups in constant time.

If you don’t want the memory overhead of a set then keep a sorted list and search through it with the bisect module.

from bisect import bisect_left
def bi_contains(lst, item):
    """ efficient `item in lst` for sorted lists """
    # if item is larger than the last its not in the list, but the bisect would 
    # find `len(lst)` as the index to insert, so check that first. Else, if the 
    # item is in the list then it has to be at index bisect_left(lst, item)
    return (item <= lst[-1]) and (lst[bisect_left(lst, item)] == item)

More Related Contents:

  1. Search list for number in range
  2. Fastest way to check if a value exists in a list
  3. Get difference between two lists
  4. Python Sets vs Lists
  5. Are tuples more efficient than lists in Python?
  6. Why is [] faster than list()?
  7. Is there a short contains function for lists?
  8. What is the time complexity of popping elements from list in Python?
  9. Python – How to check list monotonicity
  10. Finding index of an item closest to the value in a list that’s not entirely sorted
  11. [] and {} vs list() and dict(), which is better?
  12. Finding first and last index of some value in a list in Python
  13. Find an element in a list of tuples
  14. Why is tuple faster than list in Python?
  15. Complexity of list.index(x) in Python
  16. Python: get a dict from a list based on something inside the dict
  17. Searching for a string in a large text file – profiling various methods in python
  18. Create random list of integers in Python
  19. Fastest way to search a list in python
  20. Creating an empty list in Python
  21. Get difference between two lists with Unique Entries
  22. Is there a way to circumvent Python list.append() becoming progressively slower in a loop as the list grows?
  23. sum each value in a list of tuples
  24. Find starting and ending indices of sublist in list
  25. Python List Indexing Efficiency
  26. Python: iterating over list vs over dict items efficiency
  27. First items in inner list efficiently as possible [duplicate]
  28. Which is the most efficient way to iterate through a list in python?
  29. Length of longest word in a list
  30. deque.popleft() and list.pop(0). Is there performance difference?

Leave a Comment