This looks like calloc
hitting a threshold where it makes an OS request for zeroed memory and doesn’t need to initialize it manually. Looking through the source code, numpy.zeros
eventually delegates to calloc
to acquire a zeroed memory block, and if you compare to numpy.empty
, which doesn’t perform initialization:
In [15]: %timeit np.zeros((5000, 5000))
The slowest run took 12.65 times longer than the fastest. This could mean that a
n intermediate result is being cached.
100000 loops, best of 3: 10 µs per loop
In [16]: %timeit np.empty((5000, 5000))
The slowest run took 5.05 times longer than the fastest. This could mean that an
intermediate result is being cached.
100000 loops, best of 3: 10.3 µs per loop
you can see that np.zeros
has no initialization overhead for the 5000×5000 array.
In fact, the OS isn’t even “really” allocating that memory until you try to access it. A request for terabytes of array succeeds on a machine without terabytes to spare:
In [23]: x = np.zeros(2**40) # No MemoryError!