itertools product speed up

by

The NumPy equivalent of itertools.product() is numpy.indices(), but it will only get you the product of ranges of the form 0,…,k-1:

numpy.rollaxis(numpy.indices((2, 3, 3)), 0, 4)
array([[[[0, 0, 0],
         [0, 0, 1],
         [0, 0, 2]],

        [[0, 1, 0],
         [0, 1, 1],
         [0, 1, 2]],

        [[0, 2, 0],
         [0, 2, 1],
         [0, 2, 2]]],


       [[[1, 0, 0],
         [1, 0, 1],
         [1, 0, 2]],

        [[1, 1, 0],
         [1, 1, 1],
         [1, 1, 2]],

        [[1, 2, 0],
         [1, 2, 1],
         [1, 2, 2]]]])

For your special case, you can use

a = numpy.indices((4,)*13)
b = 1j ** numpy.rollaxis(a, 0, 14)

(This won’t run on a 32 bit system, because the array is to large. Extrapolating from the size I can test, it should run in less than a minute though.)

EIDT: Just to mention it: the call to numpy.rollaxis() is more or less cosmetical, to get the same output as itertools.product(). If you don’t care about the order of the indices, you can just omit it (but it is cheap anyway as long as you don’t have any follow-up operations that would transform your array into a contiguous array.)

EDIT2: To get the exact analogue of

numpy.array(list(itertools.product(some_list, repeat=some_length)))

you can use

numpy.array(some_list)[numpy.rollaxis(
    numpy.indices((len(some_list),) * some_length), 0, some_length + 1)
    .reshape(-1, some_length)]

This got completely unreadable — just tell me whether I should explain it any further 🙂

More Related Contents:

  1. N-D version of itertools.combinations in numpy
  2. Finding local maxima/minima with Numpy in a 1D numpy array
  3. Matplotlib: save plot to numpy array
  4. Find column name in pandas that matches an array
  5. Reshape an array in NumPy
  6. find length of sequences of identical values in a numpy array (run length encoding)
  7. Is it possible to vectorize recursive calculation of a NumPy array where each element depends on the previous one?
  8. Transform a set of numbers in numpy so that each number gets converted into a number of other numbers which are less than it
  9. Suppressing scientific notation in pandas? [duplicate]
  10. Python Array Slice With Comma?
  11. Python equivalent of MATLAB’s “ismember” function
  12. Linear regression with matplotlib / numpy
  13. Generate a random sample of points distributed on the surface of a unit sphere
  14. how to read certain columns from Excel using Pandas – Python
  15. How do I load a caffe model and convert to a numpy array?
  16. which is faster for load: pickle or hdf5 in python
  17. Why does numpy.linalg.solve() offer more precise matrix inversions than numpy.linalg.inv()?
  18. How to calculate cumulative normal distribution?
  19. load csv into 2D matrix with numpy for plotting
  20. Create numpy matrix filled with NaNs
  21. How to fix ‘Object arrays cannot be loaded when allow_pickle=False’ for imdb.load_data() function?
  22. Filling in date gaps in MultiIndex Pandas Dataframe
  23. Fitting a Weibull distribution using Scipy
  24. sampling random floats on a range in numpy
  25. How does the axis parameter from NumPy work?
  26. Fast interpolation of grid data
  27. numpy faster than numba and cython , how to improve numba code
  28. Shift rows of a numpy array independently
  29. Formatting floats in a numpy array [duplicate]
  30. Add density curve on the histogram

Leave a Comment