How to implement ZCA Whitening? Python

by

Here is a python function for generating the ZCA whitening matrix:

def zca_whitening_matrix(X):
    """
    Function to compute ZCA whitening matrix (aka Mahalanobis whitening).
    INPUT:  X: [M x N] matrix.
        Rows: Variables
        Columns: Observations
    OUTPUT: ZCAMatrix: [M x M] matrix
    """
    # Covariance matrix [column-wise variables]: Sigma = (X-mu)' * (X-mu) / N
    sigma = np.cov(X, rowvar=True) # [M x M]
    # Singular Value Decomposition. X = U * np.diag(S) * V
    U,S,V = np.linalg.svd(sigma)
        # U: [M x M] eigenvectors of sigma.
        # S: [M x 1] eigenvalues of sigma.
        # V: [M x M] transpose of U
    # Whitening constant: prevents division by zero
    epsilon = 1e-5
    # ZCA Whitening matrix: U * Lambda * U'
    ZCAMatrix = np.dot(U, np.dot(np.diag(1.0/np.sqrt(S + epsilon)), U.T)) # [M x M]
    return ZCAMatrix

And an example of the usage:

X = np.array([[0, 2, 2], [1, 1, 0], [2, 0, 1], [1, 3, 5], [10, 10, 10] ]) # Input: X [5 x 3] matrix
ZCAMatrix = zca_whitening_matrix(X) # get ZCAMatrix
ZCAMatrix # [5 x 5] matrix
xZCAMatrix = np.dot(ZCAMatrix, X) # project X onto the ZCAMatrix
xZCAMatrix # [5 x 3] matrix

Hope it helps!

Details for why Edgar Andrés Margffoy Tuay‘s answer is not correct: As pointed out in R.M‘s comment, Edgar Andrés Margffoy Tuay‘s ZCA whitening function contains a small, but crucial mistake: the np.diag(S) should be removed. Numpy returns S as a m x 1 vector and not a m x m matrix (as is common to other svd implementations, e.g. Matlab). Hence the ZCAMatrix variable becomes a m x 1 vector and not a m x m matrix as it should be (when the input is m x n). (Also, the covariance matrix in Andfoy’s answer is only valid if X is pre-centered, i.e mean 0).

Other references for ZCA: You can see the full answer, in Python, to the Stanford UFLDL ZCA Whitening exercise here.

More Related Contents:

  1. Principal Component Analysis (PCA) in Python
  2. Feature/Variable importance after a PCA analysis
  3. Recovering features names of explained_variance_ratio_ in PCA with sklearn
  4. Add legend to scatter plot (PCA)
  5. Principal component analysis in Python
  6. In sklearn.decomposition.PCA, why are components_ negative?
  7. Can’t pickle when using multiprocessing Pool.map()
  8. Haversine Formula in Python (Bearing and Distance between two GPS points)
  9. Convert string to Python class object?
  10. Python out of memory on large CSV file (numpy)
  11. Python read JSON file and modify
  12. Broken references in Virtualenvs
  13. How to customize title bar and window of desktop application
  14. PyPlot move alternative y axis to background
  15. Passing multiple parameters to pool.map() function in Python [duplicate]
  16. How to log source file name and line number in Python
  17. Interpolation on DataFrame in pandas
  18. MSSQL2008 – Pyodbc – Previous SQL was not a query
  19. Split list into separate but overlapping chunks
  20. Priority of the logical operators (order of operations) for NOT, AND, OR in Python
  21. Selecting options using Selenium and Python
  22. mysql-python installation problems (on mac os x lion)
  23. pygame.error: video system not initialized
  24. Faster way to remove stop words in Python
  25. How to pass a constant value to Python UDF?
  26. Generate a String that matches a RegEx in Python [duplicate]
  27. GitPython and SSH Keys?
  28. Print results in MySQL format with Python
  29. Large Performance difference when summing ints vs floats in Cython vs NumPy
  30. Fetch first 10 results from a list in Python

Leave a Comment