In sklearn.decomposition.PCA, why are components_ negative?

by

As you figured out in your answer, the results of a singular value decomposition (SVD) are not unique in terms of singular vectors. Indeed, if the SVD of X is \sum_1^r \s_i u_i v_i^\top :
enter image description here

with the s_i ordered in decreasing fashion, then you can see that you can change the sign (i.e., “flip”) of say u_1 and v_1, the minus signs will cancel so the formula will still hold.

This shows that the SVD is unique up to a change in sign in pairs of left and right singular vectors.

Since the PCA is just a SVD of X (or an eigenvalue decomposition of X^\top X), there is no guarantee that it does not return different results on the same X every time it is performed. Understandably, scikit learn implementation wants to avoid this: they guarantee that the left and right singular vectors returned (stored in U and V) are always the same, by imposing (which is arbitrary) that the largest coefficient of u_i in absolute value is positive.

As you can see reading the source: first they compute U and V with linalg.svd(). Then, for each vector u_i (i.e, row of U), if its largest element in absolute value is positive, they don’t do anything. Otherwise, they change u_i to – u_i and the corresponding left singular vector, v_i, to – v_i. As told earlier, this does not change the SVD formula since the minus sign cancel out. However, now it is guaranteed that the U and V returned after this processing are always the same, since the indetermination on the sign has been removed.

More Related Contents:

  1. Change sign of elements with an odd sum of indices
  2. difference between arrays in numpy
  3. value error in python script
  4. How do you fix “runtimeError: package fails to pass a sanity check” for numpy and pandas?
  5. Rolling window for 1D arrays in Numpy?
  6. Windows Scipy Install: No Lapack/Blas Resources Found
  7. ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
  8. Conditionally fill column values based on another columns value in pandas
  9. How to normalize a NumPy array to a unit vector?
  10. Principal Component Analysis (PCA) in Python
  11. Python’s sum vs. NumPy’s numpy.sum
  12. How to do n-D distance and nearest neighbor calculations on numpy arrays
  13. numpy is already installed with Anaconda but I get an ImportError (DLL load failed: The specified module could not be found)
  14. Feature/Variable importance after a PCA analysis
  15. scipy curve_fit doesn’t like math module
  16. Recovering features names of explained_variance_ratio_ in PCA with sklearn
  17. Is there anything faster than dict()?
  18. Type hinting / annotation (PEP 484) for numpy.ndarray
  19. Custom transformer for sklearn Pipeline that alters both X and y
  20. Scikit Learn SVC decision_function and predict
  21. RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility
  22. Error “Import Error: No module named numpy” on Windows
  23. LogisticRegression: Unknown label type: ‘continuous’ using sklearn in python
  24. pandas.read_csv from string or package data
  25. Principal component analysis in Python
  26. Python pandas: how to remove nan and -inf values
  27. How to use sklearn fit_transform with pandas and return dataframe instead of numpy array?
  28. AttributeError: module ‘numpy’ has no attribute ‘__version__’
  29. Python import error: cannot import name ‘six’ from ‘sklearn.externals’
  30. ValueError: Dimension mismatch

Leave a Comment