Why “numpy.any” has no short-circuit mechanism?

by

It’s an unfixed performance regression. NumPy issue 3446. There actually is short-circuiting logic, but a change to the ufunc.reduce machinery introduced an unnecessary chunk-based outer loop around the short-circuiting logic, and that outer loop doesn’t know how to short circuit. You can see some explanation of the chunking machinery here.

The short-circuiting effects wouldn’t have showed up in your test even without the regression, though. First, you’re timing the array creation, and second, I don’t think they ever put in the short-circuit logic for any input dtype but boolean. From the discussion, it sounds like the details of the ufunc reduction machinery behind numpy.any would have made that difficult.

The discussion does bring up the surprising point that the argmin and argmax methods appear to short-circuit for boolean input. A quick test shows that as of NumPy 1.12 (not quite the most recent version, but the version currently on Ideone), x[x.argmax()] short-circuits, and it outcompetes x.any() and x.max() for 1-dimensional boolean input no matter whether the input is small or large and no matter whether the short-circuiting pays off. Weird!

More Related Contents:

  1. numpy: most efficient frequency counts for unique values in an array
  2. Why is numpy’s einsum faster than numpy’s built in functions?
  3. Roll rows of a matrix independently
  4. Numpy sum elements in array based on its value
  5. Efficiently return the index of the first value satisfying condition in array
  6. Link ATLAS/MKL to an installed Numpy
  7. Bin elements per row – Vectorized 2D Bincount for NumPy
  8. Fastest way to grow a numpy numeric array
  9. Vectorized searchsorted numpy
  10. Most efficient way to forward-fill NaN values in numpy array
  11. Numpy: Fix array with rows of different lengths by filling the empty elements with zeros
  12. Vectorizing Haversine distance calculation in Python
  13. Numpy and line intersections
  14. Why is numpy.array so slow?
  15. Efficient evaluation of a function at every cell of a NumPy array
  16. why isn’t numpy.mean multithreaded?
  17. Why list comprehension is much faster than numpy for multiplying arrays?
  18. Why is `np.sum(range(N))` very slow?
  19. Why is a `for` over a Python list faster than over a Numpy array?
  20. Performance: Matlab vs Python
  21. Are NumPy’s math functions faster than Python’s?
  22. Efficiently replace elements in array based on dictionary – NumPy / Python
  23. Why is Numpy much faster at creating a Zero array compared to replacing the values of an existing array with zeros?
  24. Getting days since last occurence in Pandas DataFrame?
  25. Multiple matrix multiplication
  26. What is the most efficient way to check if a value exists in a NumPy array?
  27. Efficient dot products of large memory-mapped arrays
  28. Creating a numpy array of 3D coordinates from three 1D arrays
  29. Split Python sequence (time series/array) into subsequences with overlap
  30. numpy faster than numba and cython , how to improve numba code

Leave a Comment