How does condensed distance matrix work? (pdist)

by

You can look at it this way: Suppose x is m by n. The possible pairs of m rows, chosen two at a time, is itertools.combinations(range(m), 2), e.g, for m=3:

>>> import itertools
>>> list(combinations(range(3),2))
[(0, 1), (0, 2), (1, 2)]

So if d = pdist(x), the kth tuple in combinations(range(m), 2)) gives the indices of the rows of x associated with d[k].

Example:

>>> x = array([[0,10],[10,10],[20,20]])
>>> pdist(x)
array([ 10.        ,  22.36067977,  14.14213562])

The first element is dist(x[0], x[1]), the second is dist(x[0], x[2]) and the third is dist(x[1], x[2]).

Or you can view it as the elements in the upper triangular part of the square distance matrix, strung together into a 1D array.

E.g.

>>> squareform(pdist(x)) 
array([[  0.   ,  10.   ,  22.361],
       [ 10.   ,   0.   ,  14.142],
       [ 22.361,  14.142,   0.   ]])

>>> y = array([[0,10],[10,10],[20,20],[10,0]])
>>> squareform(pdist(y)) 
array([[  0.   ,  10.   ,  22.361,  14.142],
       [ 10.   ,   0.   ,  14.142,  10.   ],
       [ 22.361,  14.142,   0.   ,  22.361],
       [ 14.142,  10.   ,  22.361,   0.   ]])
>>> pdist(y)
array([ 10.   ,  22.361,  14.142,  14.142,  10.   ,  22.361])

More Related Contents:

  1. How do I read CSV data into a record array in NumPy?
  2. Working with big data in python and numpy, not enough ram, how to save partial results on disc?
  3. Python out of memory on large CSV file (numpy)
  4. Histogram Matplotlib
  5. python numpy/scipy curve fitting
  6. Multiple linear regression in Python
  7. Plotting a fast Fourier transform in Python
  8. How to check BLAS/LAPACK linkage in NumPy and SciPy?
  9. How to create a density plot in matplotlib?
  10. How to do n-D distance and nearest neighbor calculations on numpy arrays
  11. Fitting a Normal distribution to 1D data
  12. Populate a Pandas SparseDataFrame from a SciPy Sparse Matrix
  13. How to extend an array in-place in Numpy?
  14. How do I install SciPy on 64 bit Windows?
  15. Removing duplicate columns and rows from a NumPy 2D array
  16. Numpy Resize/Rescale Image
  17. How to specify upper and lower limits when using numpy.random.normal
  18. fitting exponential decay with no initial guessing
  19. How to plot line (polygonal chain) with numpy/scipy/matplotlib with minimal smoothing
  20. Storing numpy sparse matrix in HDF5 (PyTables)
  21. How to calculate the statistics “t-test” with numpy
  22. How do I use numpy.where()? What should I pass, and what does the result mean? [closed]
  23. Creating a Confidence Ellipsis in a scatterplot using matplotlib
  24. Load just part of an image in python
  25. generalized cumulative functions in NumPy/SciPy?
  26. Vectorized spatial distance in python using numpy
  27. Principal component analysis in Python
  28. Numpy transpose of 1D array not giving expected result
  29. Fit mixture of two gaussian/normal distributions to a histogram from one set of data, python
  30. Speedup scipy griddata for multiple interpolations between two irregular grids

Leave a Comment