Integer overflow in numpy arrays

Solution 1:

On your platform, np.arange returns an array of dtype 'int32' :

In [1]: np.arange(1000000).dtype
Out[1]: dtype('int32')

Each element of the array is a 32-bit integer. Squaring leads to a result which does not fit in 32-bits. The result is cropped to 32-bits and still interpreted as a 32-bit integer, however, which is why you see negative numbers.

Edit: In this case, you can avoid the integer overflow by constructing an array of dtype 'int64' before squaring:

a=np.arange(1000000,dtype='int64').reshape(1000,1000)

Note that the problem you've discovered is an inherent danger when working with numpy. You have to choose your dtypes with care and know before-hand that your code will not lead to arithmetic overflows. For the sake of speed, numpy can not and will not warn you when this occurs.

See http://mail.scipy.org/pipermail/numpy-discussion/2009-April/041691.html for a discussion of this on the numpy mailing list.

Solution 2:

python integers don't have this problem, since they automatically upgrade to python long integers when they overflow.

so if you do manage to overflow the int64's, one solution is to use python int's in the numpy array:

import numpy
a=numpy.arange(1000,dtype=object)
a**20

Solution 3:

numpy integer types are fixed width and you are seeing the results of integer overflow.