How to create a numpy array of arbitrary length strings?

by

You can do so by creating an array of dtype=object. If you try to assign a long string to a normal numpy array, it truncates the string:

>>> a = numpy.array(['apples', 'foobar', 'cowboy'])
>>> a[2] = 'bananas'
>>> a
array(['apples', 'foobar', 'banana'], 
      dtype="|S6")

But when you use dtype=object, you get an array of python object references. So you can have all the behaviors of python strings:

>>> a = numpy.array(['apples', 'foobar', 'cowboy'], dtype=object)
>>> a
array([apples, foobar, cowboy], dtype=object)
>>> a[2] = 'bananas'
>>> a
array([apples, foobar, bananas], dtype=object)

Indeed, because it’s an array of objects, you can assign any kind of python object to the array:

>>> a[2] = {1:2, 3:4}
>>> a
array([apples, foobar, {1: 2, 3: 4}], dtype=object)

However, this undoes a lot of the benefits of using numpy, which is so fast because it works on large contiguous blocks of raw memory. Working with python objects adds a lot of overhead. A simple example:

>>> a = numpy.array(['abba' for _ in range(10000)])
>>> b = numpy.array(['abba' for _ in range(10000)], dtype=object)
>>> %timeit a.copy()
100000 loops, best of 3: 2.51 us per loop
>>> %timeit b.copy()
10000 loops, best of 3: 48.4 us per loop

More Related Contents:

  1. Element-wise string concatenation in numpy
  2. Setting the fmt option in numpy.savetxt
  3. How can I slice each element of a numpy array of strings?
  4. Sorting arrays in NumPy by column
  5. How to access the ith column of a NumPy multidimensional array?
  6. Python/NumPy first occurrence of subarray
  7. Concatenating two one-dimensional NumPy arrays
  8. Numpy matrix to array
  9. Replacing Numpy elements if condition is met
  10. How to make a multidimension numpy array with a varying row size?
  11. How can I add new dimensions to a Numpy array?
  12. Force NumPy ndarray to take ownership of its memory in Cython
  13. How to normalize a NumPy array to within a certain range?
  14. Selecting specific rows and columns from NumPy array
  15. subsampling every nth entry in a numpy array
  16. How to get a list of all indices of repeated elements in a numpy array
  17. Binary random array with a specific proportion of ones?
  18. List to array conversion to use ravel() function
  19. Compress numpy arrays efficiently
  20. Add multiple values to one numpy array index
  21. Index 2D numpy array by a 2D array of indices without loops
  22. Flattening a list of NumPy arrays?
  23. Fastest pairwise distance metric in python
  24. NumPy array/matrix of mixed types
  25. Partition array into N chunks with Numpy
  26. Numpy find indices of groups with same value
  27. Numpy Vector (N,1) dimension -> (N,) dimension conversion
  28. Efficient dot products of large memory-mapped arrays
  29. Fast string array – Cython
  30. Mapping a NumPy array in place

Leave a Comment