Iterating through a scipy.sparse vector (or matrix)

by

Edit: bbtrb’s method (using coo_matrix) is much faster than my original suggestion, using nonzero. Sven Marnach’s suggestion to use itertools.izip also improves the speed. Current fastest is using_tocoo_izip:

import scipy.sparse
import random
import itertools

def using_nonzero(x):
    rows,cols = x.nonzero()
    for row,col in zip(rows,cols):
        ((row,col), x[row,col])

def using_coo(x):
    cx = scipy.sparse.coo_matrix(x)    
    for i,j,v in zip(cx.row, cx.col, cx.data):
        (i,j,v)

def using_tocoo(x):
    cx = x.tocoo()    
    for i,j,v in zip(cx.row, cx.col, cx.data):
        (i,j,v)

def using_tocoo_izip(x):
    cx = x.tocoo()    
    for i,j,v in itertools.izip(cx.row, cx.col, cx.data):
        (i,j,v)

N=200
x = scipy.sparse.lil_matrix( (N,N) )
for _ in xrange(N):
    x[random.randint(0,N-1),random.randint(0,N-1)]=random.randint(1,100)

yields these timeit results:

% python -mtimeit -s'import test' 'test.using_tocoo_izip(test.x)'
1000 loops, best of 3: 670 usec per loop
% python -mtimeit -s'import test' 'test.using_tocoo(test.x)'
1000 loops, best of 3: 706 usec per loop
% python -mtimeit -s'import test' 'test.using_coo(test.x)'
1000 loops, best of 3: 802 usec per loop
% python -mtimeit -s'import test' 'test.using_nonzero(test.x)'
100 loops, best of 3: 5.25 msec per loop

More Related Contents:

  1. Sparse matrix slicing using list of int
  2. Populate a Pandas SparseDataFrame from a SciPy Sparse Matrix
  3. Efficiently create sparse pivot tables in pandas?
  4. Pandas sparse dataFrame to sparse matrix, without generating a dense matrix in memory
  5. Directly use Intel mkl library on Scipy sparse matrix to calculate A dot A.T with less memory
  6. Concatenate sparse matrices in Python using SciPy/Numpy
  7. Scipy.sparse.csr_matrix: How to get top ten values and indices?
  8. How to transform numpy.matrix or array to scipy sparse matrix
  9. sparse 3d matrix/array in Python?
  10. Sorting arrays in NumPy by column
  11. Moving average or running mean
  12. How to normalize a NumPy array to within a certain range?
  13. Can’t install Scipy through pip
  14. Colorize Voronoi Diagram
  15. Find matching rows in 2 dimensional numpy array
  16. Resampling a numpy array representing an image
  17. Most efficient way to sort an array into bins specified by an index array?
  18. ERROR: Could not build wheels for scipy which use PEP 517 and cannot be installed directly
  19. how to annotate heatmap with text in matplotlib
  20. Fit a gaussian function
  21. AttributeError: ‘module’ object (scipy) has no attribute ‘misc’
  22. Linear regression of arrays containing NANs in Python/Numpy
  23. How to filter numpy array by list of indices?
  24. Specify format of floats for tick labels
  25. Fitting a closed curve to a set of points
  26. Multiple variables in SciPy’s optimize.minimize
  27. How to multiply two 2D RFFT arrays (FFTPACK) to be compatible with NumPy’s FFT?
  28. Compute a confidence interval from sample data assuming unknown distribution
  29. Is the order of a Python dictionary guaranteed over iterations?
  30. scipy.optimize.leastsq with bound constraints

Leave a Comment