Integer overflow in numpy arrays

by

On your platform, np.arange returns an array of dtype ‘int32’ :

In [1]: np.arange(1000000).dtype
Out[1]: dtype('int32')

Each element of the array is a 32-bit integer. Squaring leads to a result which does not fit in 32-bits. The result is cropped to 32-bits and still interpreted as a 32-bit integer, however, which is why you see negative numbers.

Edit: In this case, you can avoid the integer overflow by constructing an array of dtype ‘int64’ before squaring:

a=np.arange(1000000,dtype="int64").reshape(1000,1000)

Note that the problem you’ve discovered is an inherent danger when working with numpy. You have to choose your dtypes with care and know before-hand that your code will not lead to arithmetic overflows. For the sake of speed, numpy can not and will not warn you when this occurs.

See http://mail.scipy.org/pipermail/numpy-discussion/2009-April/041691.html for a discussion of this on the numpy mailing list.

More Related Contents:

  1. Binning a column with Python Pandas
  2. Fitting empirical distribution to theoretical ones with Scipy (Python)?
  3. ValueError: setting an array element with a sequence
  4. Numpy `logical_or` for more than two arguments
  5. Simple Digit Recognition OCR in OpenCV-Python
  6. surface plots in matplotlib
  7. Best way to join / merge by range in pandas
  8. How to install python modules without root access?
  9. How do you use the ellipsis slicing syntax in Python?
  10. Fastest way to grow a numpy numeric array
  11. Concatenate two NumPy arrays vertically
  12. How to make scipy.interpolate give an extrapolated result beyond the input range?
  13. Inverse Distance Weighted (IDW) Interpolation with Python
  14. Convert NumPy array to Python list
  15. Minimum Euclidean distance between points in two different Numpy arrays, not within
  16. Increment Numpy array with repeated indices
  17. Why “numpy.any” has no short-circuit mechanism?
  18. Use scipy.integrate.quad to integrate complex numbers
  19. Set values on the diagonal of pandas.DataFrame
  20. numpy.array.__iadd__ and repeated indices [duplicate]
  21. Polar contour plot in matplotlib – best (modern) way to do it?
  22. RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  23. Why is a `for` over a Python list faster than over a Numpy array?
  24. How do I convert an OpenCV image (BGR and BGRA) to a pygame.Surface object
  25. Why are frequency values rounded in signal using FFT?
  26. Is there a numpy builtin to reject outliers from a list
  27. How to determine whether a column/variable is numeric or not in Pandas/NumPy?
  28. Why does numpy have a corresponding function for many ndarray methods?
  29. Python OpenCV2 (cv2) wrapper to get image size?
  30. Elegant way to perform tuple arithmetic

Leave a Comment