Why is Numpy much faster at creating a Zero array compared to replacing the values of an existing array with zeros?

by

The reason is that the array is not filled in memory on mainstream operating systems (Windows, Linux and MaxOS). Numpy allocates a zero-filled array by requesting to the operating systems (OS) a zero-filled area in virtual memory. This area is not directly mapping in physical RAM. The mapping and zero-initialization is generally done lazily by the OS when you read/write the pages in virtual memory. This cost is paid when you set later the array to 1 for example. Here is a proof:

In [19]: %timeit res = np.zeros(shape=(2500, 1700))
10.8 µs ± 118 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

In [20]: %timeit res = np.ones(shape=(2500, 1700))
7.54 ms ± 151 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

The former would imply a RAM throughput of at least 4.2 GiB/s which is not high but fair. The latter would imply a RAM throughput of at least roughly 2930 GiB/s which is stupidly high since my machine (as well as any standard desktop/server machine) is barely able to reach 36 GiB/s (using a carefully-optimized benchmark).

More Related Contents:

  1. Performance of Pandas apply vs np.vectorize to create new column from existing columns
  2. numpy: most efficient frequency counts for unique values in an array
  3. Why is numpy’s einsum faster than numpy’s built in functions?
  4. Numpy sum elements in array based on its value
  5. Efficiently return the index of the first value satisfying condition in array
  6. Frequency counts for unique values in a NumPy array
  7. Most efficient way to forward-fill NaN values in numpy array
  8. Numpy: Fix array with rows of different lengths by filling the empty elements with zeros
  9. Why is a `for` over a Python list faster than over a Numpy array?
  10. Fastest save and load options for a numpy array
  11. Efficient dot products of large memory-mapped arrays
  12. Creating a numpy array of 3D coordinates from three 1D arrays
  13. How to print the full NumPy array, without truncation?
  14. Convert Python sequence to NumPy array, filling missing values
  15. Numpy array assignment with copy
  16. numpy matrix vector multiplication [duplicate]
  17. Repeating each element of a numpy array 5 times
  18. how does multiplication differ for NumPy Matrix vs Array classes?
  19. How to split/partition a dataset into training and test datasets for, e.g., cross validation?
  20. How to create a numpy array of arbitrary length strings?
  21. Numpy modify array in place?
  22. Creating same random number sequence in Python, NumPy and R
  23. pd.Timestamp versus np.datetime64: are they interchangeable for selected uses?
  24. Prevent numpy from creating a multidimensional array
  25. Numpy – Replace a number with NaN
  26. Python: how to store a numpy multidimensional array in PyTables?
  27. Numpy shuffle multidimensional array by row only, keep column order unchanged
  28. Randomly select from numpy array
  29. How to “scale” a numpy array?
  30. Generate random array of floats between a range

Leave a Comment