How do I determine k when using k-means clustering?

by

You can maximize the Bayesian Information Criterion (BIC):

BIC(C | X) = L(X | C) - (p / 2) * log n

where L(X | C) is the log-likelihood of the dataset X according to model C, p is the number of parameters in the model C, and n is the number of points in the dataset.
See “X-means: extending K-means with efficient estimation of the number of clusters” by Dan Pelleg and Andrew Moore in ICML 2000.

Another approach is to start with a large value for k and keep removing centroids (reducing k) until it no longer reduces the description length. See “MDL principle for robust vector quantisation” by Horst Bischof, Ales Leonardis, and Alexander Selb in Pattern Analysis and Applications vol. 2, p. 59-72, 1999.

Finally, you can start with one cluster, then keep splitting clusters until the points assigned to each cluster have a Gaussian distribution. In “Learning the k in k-means” (NIPS 2003), Greg Hamerly and Charles Elkan show some evidence that this works better than BIC, and that BIC does not penalize the model’s complexity strongly enough.

More Related Contents:

  1. Cluster analysis in R: determine the optimal number of clusters
  2. Is it possible to specify your own distance function using scikit-learn K-Means Clustering?
  3. Clustering text documents using scikit-learn kmeans in Python
  4. Cluster one-dimensional data optimally? [closed]
  5. K-means algorithm variation with equal cluster size
  6. Group n points in k clusters of equal size [duplicate]
  7. python kmeans on string
  8. An out-of-memory error in clustering
  9. How do I know which observations belong to which cluster? [closed]
  10. 1D Number Array Clustering
  11. scikit-learn DBSCAN memory usage
  12. plotting results of hierarchical clustering ontop of a matrix of data in python
  13. Reading wav file in Java
  14. Unsupervised clustering with unknown number of clusters
  15. Text clustering with Levenshtein distances
  16. Is Spark’s KMeans unable to handle bigdata?
  17. plotting results of hierarchical clustering on top of a matrix of data
  18. Clustering values by their proximity in python (machine learning?) [duplicate]
  19. Simple approach to assigning clusters for new data after k-means clustering
  20. Clustering Algorithm for Mapping Application
  21. Scikit Learn GridSearchCV without cross validation (unsupervised learning)
  22. Understanding concept of Gaussian Mixture Models
  23. How to optimal K in K – Means Algorithm [duplicate]
  24. Identifying points by color
  25. How to get flat clustering corresponding to color clusters in the dendrogram created by scipy

Leave a Comment