Python built-in sum function vs. for loop performance

by

The speed difference is actually greater than 3 times, but you slow down either version by first creating a huge in-memory list of 1 million integers. Separate that out of the time trials:

>>> import timeit
>>> def sum1(lst):
...     s = 0
...     for i in lst:
...         s += i
...     return s
... 
>>> def sum2(lst):
...     return sum(lst)
... 
>>> values = range(1000000)
>>> timeit.timeit('f(lst)', 'from __main__ import sum1 as f, values as lst', number=100)
3.457869052886963
>>> timeit.timeit('f(lst)', 'from __main__ import sum2 as f, values as lst', number=100)
0.6696369647979736

The speed difference has risen to over 5 times now.

A for loop is executed as interpreted Python bytecode. sum() loops entirely in C code. The speed difference between interpreted bytecode and C code is large.

In addition, the C code makes sure not to create new Python objects if it can keep the sum in C types instead; this works for int and float results.

The Python version, disassembled, does this:

>>> import dis
>>> def sum1():
...     s = 0
...     for i in range(1000000):
...         s += i
...     return s
... 
>>> dis.dis(sum1)
  2           0 LOAD_CONST               1 (0)
              3 STORE_FAST               0 (s)

  3           6 SETUP_LOOP              30 (to 39)
              9 LOAD_GLOBAL              0 (range)
             12 LOAD_CONST               2 (1000000)
             15 CALL_FUNCTION            1
             18 GET_ITER            
        >>   19 FOR_ITER                16 (to 38)
             22 STORE_FAST               1 (i)

  4          25 LOAD_FAST                0 (s)
             28 LOAD_FAST                1 (i)
             31 INPLACE_ADD         
             32 STORE_FAST               0 (s)
             35 JUMP_ABSOLUTE           19
        >>   38 POP_BLOCK           

  5     >>   39 LOAD_FAST                0 (s)
             42 RETURN_VALUE

Apart from the interpreter loop being slower than C, the INPLACE_ADD will create a new integer object (past 255, CPython caches small int objects as singletons).

You can see the C implementation in the Python mercurial code repository, but it explicitly states in the comments:

/* Fast addition by keeping temporary sums in C instead of new Python objects.
   Assumes all inputs are the same type.  If the assumption fails, default
   to the more general routine.
*/

More Related Contents:

  1. Is doing multiplication multiple times or assigning to a variable faster?
  2. Sum the digits of a number
  3. pandas loc vs. iloc vs. at vs. iat?
  4. Pandas: sum DataFrame rows for given columns
  5. numpy: most efficient frequency counts for unique values in an array
  6. What is the best way to generate all possible three letter strings?
  7. Why is [] faster than list()?
  8. Why is numpy’s einsum faster than numpy’s built in functions?
  9. Numpy sum elements in array based on its value
  10. Efficiently return the index of the first value satisfying condition in array
  11. Pandas counting and summing specific conditions
  12. Speed comparison with Project Euler: C vs Python vs Erlang vs Haskell
  13. What’s a faster operation, re.match/search or str.find?
  14. How do I measure elapsed time in Python?
  15. Cost of exception handlers in Python
  16. Sum of list of lists; returns sum list
  17. File read using “open()” vs “with open()” [duplicate]
  18. It is more efficient to use if-return-return or if-else-return?
  19. Most efficient way to forward-fill NaN values in numpy array
  20. [] and {} vs list() and dict(), which is better?
  21. python: deque vs list performance comparison
  22. multiprocessing.Pool() slower than just using ordinary functions
  23. Efficiently Calculating a Euclidean Distance Matrix Using Numpy
  24. Is & faster than % when checking for odd numbers?
  25. What is the fastest way to output large DataFrame into a CSV file?
  26. How expensive are Python dictionaries to handle?
  27. differences between “d = dict()” and “d = {}”
  28. Creating a numpy array of 3D coordinates from three 1D arrays
  29. Split Python sequence (time series/array) into subsequences with overlap
  30. numpy faster than numba and cython , how to improve numba code

Leave a Comment