How to create a fix size list in python?

by

(tl;dr: The exact answer to your question is numpy.empty or numpy.empty_like, but you likely don’t care and can get away with using myList = [None]*10000.)

Simple methods

You can initialize your list to all the same element. Whether it semantically makes sense to use a non-numeric value (that will give an error later if you use it, which is a good thing) or something like 0 (unusual? maybe useful if you’re writing a sparse matrix or the ‘default’ value should be 0 and you’re not worried about bugs) is up to you:

>>> [None for _ in range(10)]
[None, None, None, None, None, None, None, None, None, None]

(Here _ is just a variable name, you could have used i.)

You can also do so like this:

>>> [None]*10
[None, None, None, None, None, None, None, None, None, None]

You probably don’t need to optimize this. You can also append to the array every time you need to:

>>> x = []
>>> for i in range(10):
>>>    x.append(i)

Performance comparison of simple methods

Which is best?

>>> def initAndWrite_test():
...  x = [None]*10000
...  for i in range(10000):
...   x[i] = i
... 
>>> def initAndWrite2_test():
...  x = [None for _ in range(10000)]
...  for i in range(10000):
...   x[i] = i
... 
>>> def appendWrite_test():
...  x = []
...  for i in range(10000):
...   x.append(i)

Results in python2.7:

>>> import timeit
>>> for f in [initAndWrite_test, initAndWrite2_test, appendWrite_test]:
...  print('{} takes {} usec/loop'.format(f.__name__, timeit.timeit(f, number=1000)*1000))
... 
initAndWrite_test takes 714.596033096 usec/loop
initAndWrite2_test takes 981.526136398 usec/loop
appendWrite_test takes 908.597946167 usec/loop

Results in python 3.2:

initAndWrite_test takes 641.3581371307373 usec/loop
initAndWrite2_test takes 1033.6499214172363 usec/loop
appendWrite_test takes 895.9040641784668 usec/loop

As we can see, it is likely better to do the idiom [None]*10000 in both python2 and python3. However, if one is doing anything more complicated than assignment (such as anything complicated to generate or process every element in the list), then the overhead becomes a meaninglessly small fraction of the cost. That is, such optimization is premature to worry about if you’re doing anything reasonable with the elements of your list.

Uninitialized memory

These are all however inefficient because they go through memory, writing something in the process. In C this is different: an uninitialized array is filled with random garbage memory (sidenote: that has been reallocated from the system, and can be a security risk when you allocate or fail to mlock and/or fail to delete memory when closing the program). This is a design choice, designed for speedup: the makers of the C language thought that it was better not to automatically initialize memory, and that was the correct choice.

This is not an asymptotic speedup (because it’s O(N)), but for example you wouldn’t need to first initialize your entire memory block before you overwrite with stuff you actually care about. This, if it were possible, is equivalent to something like (pseudo-code) x = list(size=10000).

If you want something similar in python, you can use the numpy numerical matrix/N-dimensional-array manipulation package. Specifically, numpy.empty or numpy.empty_like

That is the real answer to your question.

More Related Contents:

  1. How do i search for 2 of the same numbers in a list? (python) [closed]
  2. Unable to use convert strings into integers for a for loop
  3. How to remove an element from a list by index
  4. List of lists changes reflected across sublists unexpectedly
  5. Strange result when removing item from a list while iterating over it
  6. How to merge lists into a list of tuples?
  7. How can I compare two lists in python and return matches
  8. TypeError: ‘list’ object is not callable in python
  9. Difference between del, remove, and pop on lists
  10. Pythonic way to check if a list is sorted or not
  11. Python: Finding differences between elements of a list
  12. List comprehension vs. lambda + filter
  13. Python – use list as function parameters
  14. Append to a list defined in a tuple – is it a bug? [duplicate]
  15. String to list in Python
  16. How to extract parameters from a list and pass them to a function call [duplicate]
  17. Generate a list of datetimes between an interval
  18. case-insensitive list sorting, without lowercasing the result?
  19. python assign values to list elements in loop
  20. How to add element in Python to the end of list using list.insert?
  21. Append integer to beginning of list in Python [duplicate]
  22. Efficient method to calculate the rank vector of a list in Python
  23. Duplicate each member in a list [duplicate]
  24. How do I reverse a list using recursion in Python?
  25. python – get list of tuples first index?
  26. Python: Elegant and efficient ways to mask a list
  27. Creating a list in Python- something sneaky going on?
  28. Rearranging list based on order of another list
  29. list comprehension with multiple conditions (python)
  30. How to convert only numbers in a mixed list into float

Leave a Comment