Efficiently replace elements in array based on dictionary – NumPy / Python

by

Approach #1 : Loopy one with array data

One approach would be extracting the keys and values in arrays and then use a similar loop –

k = np.array(list(mapping.keys()))
v = np.array(list(mapping.values()))

out = np.zeros_like(input_array)
for key,val in zip(k,v):
    out[input_array==key] = val

Benefit with this one over the original one is the spatial-locality of the array data for efficient data-fetching, which is used in the iterations.

Also, since you mentioned thousand large np.arrays. So, if the mapping dictionary stays the same, that step to get the array versions – k and v would be a one-time setup process.

Approach #2 : Vectorized one with searchsorted

A vectorized one could be suggested using np.searchsorted –

sidx = k.argsort() #k,v from approach #1

k = k[sidx]
v = v[sidx]

idx = np.searchsorted(k,input_array.ravel()).reshape(input_array.shape)
idx[idx==len(k)] = 0
mask = k[idx] == input_array
out = np.where(mask, v[idx], 0)

Approach #3 : Vectorized one with mapping-array for integer keys

A vectorized one could be suggested using a mapping array for integer keys, which when indexed by the input array would lead us directly to the final output –

mapping_ar = np.zeros(k.max()+1,dtype=v.dtype) #k,v from approach #1
mapping_ar[k] = v
out = mapping_ar[input_array]

More Related Contents:

  1. Bin elements per row – Vectorized 2D Bincount for NumPy
  2. Vectorized searchsorted numpy
  3. Vectorizing Haversine distance calculation in Python
  4. Efficient evaluation of a function at every cell of a NumPy array
  5. Taking subarrays from numpy array with given stride/stepsize
  6. Roll rows of a matrix independently
  7. Link ATLAS/MKL to an installed Numpy
  8. Fastest way to grow a numpy numeric array
  9. Implement MATLAB’s im2col ‘sliding’ in Python
  10. Python’s sum vs. NumPy’s numpy.sum
  11. Why “numpy.any” has no short-circuit mechanism?
  12. Using Numpy Vectorize on Functions that Return Vectors
  13. Vectorized look-up of values in Pandas dataframe
  14. Python vectorizing nested for loops
  15. Why is numpy.array so slow?
  16. why isn’t numpy.mean multithreaded?
  17. Why is `np.sum(range(N))` very slow?
  18. Why is a `for` over a Python list faster than over a Numpy array?
  19. vectorize conditional assignment in pandas dataframe
  20. Subtracting numpy arrays of different shape efficiently
  21. Performance: Matlab vs Python
  22. Are NumPy’s math functions faster than Python’s?
  23. Concatenate range arrays given start, stop numbers in a vectorized way – NumPy
  24. Why is Numpy much faster at creating a Zero array compared to replacing the values of an existing array with zeros?
  25. Multiple matrix multiplication
  26. Creating a numpy array of 3D coordinates from three 1D arrays
  27. Vectorized moving window on 2D array in numpy
  28. Split Python sequence (time series/array) into subsequences with overlap
  29. Vectorizing `numpy.random.choice` for given 2D array of probabilities along an axis
  30. numpy faster than numba and cython , how to improve numba code

Leave a Comment