Clustering text documents using scikit-learn kmeans in Python

by

This is a simpler example:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score

documents = ["Human machine interface for lab abc computer applications",
             "A survey of user opinion of computer system response time",
             "The EPS user interface management system",
             "System and human system engineering testing of EPS",
             "Relation of user perceived response time to error measurement",
             "The generation of random binary unordered trees",
             "The intersection graph of paths in trees",
             "Graph minors IV Widths of trees and well quasi ordering",
             "Graph minors A survey"]

vectorize the text i.e. convert the strings to numeric features

vectorizer = TfidfVectorizer(stop_words="english")
X = vectorizer.fit_transform(documents)

cluster documents

true_k = 2
model = KMeans(n_clusters=true_k, init="k-means++", max_iter=100, n_init=1)
model.fit(X)

print top terms per cluster clusters

print("Top terms per cluster:")
order_centroids = model.cluster_centers_.argsort()[:, ::-1]
terms = vectorizer.get_feature_names()
for i in range(true_k):
    print "Cluster %d:" % i,
    for ind in order_centroids[i, :10]:
        print ' %s' % terms[ind],
    print

If you want to have a more visual idea of how this looks like see this answer.

More Related Contents:

  1. Is it possible to specify your own distance function using scikit-learn K-Means Clustering?
  2. python kmeans on string
  3. sklearn error ValueError: Input contains NaN, infinity or a value too large for dtype(‘float64’)
  4. scikit-learn DBSCAN memory usage
  5. Scikit Learn GridSearchCV without cross validation (unsupervised learning)
  6. TypeError: cannot perform reduce with flexible type
  7. confused about random_state in decision tree of scikit learn
  8. how to parse the response in python?
  9. How to add a label to all words in a file? [closed]
  10. Convert string to variable name in python [duplicate]
  11. What do ellipsis […] mean in a list?
  12. Pelican 3.3 pelican-quickstart error “ValueError: unknown locale: UTF-8”
  13. How to get FTP file’s modify time using Python ftplib
  14. memoization library for python 2.7
  15. How to get precision, recall and f-measure from confusion matrix in Python [duplicate]
  16. Passing data between separately running Python scripts
  17. How do I run python 2 and 3 in windows 7? [duplicate]
  18. Custom transformer for sklearn Pipeline that alters both X and y
  19. Is Python’s order of evaluation of function arguments and operands deterministic (+ where is it documented)?
  20. Save MinMaxScaler model in sklearn
  21. Get Excel-Style Column Names from Column Number
  22. confusion matrix error “Classification metrics can’t handle a mix of multilabel-indicator and multiclass targets”
  23. What are the pros and cons between get_dummies (Pandas) and OneHotEncoder (Scikit-learn)?
  24. Run an OLS regression with Pandas Data Frame
  25. Group DataFrame in 5-minute intervals
  26. How to draw rectangle outside of the plot frame in Matplotlib
  27. Python import error: cannot import name ‘six’ from ‘sklearn.externals’
  28. Python: How to resize an image using PIL module
  29. Python not able to open file with non-english characters in path
  30. How to fix TypeError: ‘int’ object is not subscriptable

Leave a Comment