TfidfVectorizer in scikit-learn : ValueError: np.nan is an invalid document

by

You need to convert the dtype object to unicode string as is clearly mentioned in the traceback.

x = v.fit_transform(df['Review'].values.astype('U'))  ## Even astype(str) would work

From the Doc page of TFIDF Vectorizer:

fit_transform(raw_documents, y=None)

Parameters: raw_documents : iterable
an iterable which yields either str, unicode or file objects

More Related Contents:

  1. Custom transformer for sklearn Pipeline that alters both X and y
  2. Scikit Learn OneHotEncoder fit and transform Error: ValueError: X has different shape than during fitting
  3. What are the pros and cons between get_dummies (Pandas) and OneHotEncoder (Scikit-learn)?
  4. Label encoding across multiple columns in scikit-learn
  5. Save classifier to disk in scikit-learn
  6. How to get most informative features for scikit-learn classifiers?
  7. Dummy variables when not all categories are present
  8. Python: tf-idf-cosine: to find document similarity
  9. Why is Random Forest with a single tree much better than a Decision Tree classifier?
  10. How to get precision, recall and f-measure from confusion matrix in Python [duplicate]
  11. How to convert a Scikit-learn dataset to a Pandas dataset
  12. Feature/Variable importance after a PCA analysis
  13. Recovering features names of explained_variance_ratio_ in PCA with sklearn
  14. What is exactly sklearn.pipeline.Pipeline?
  15. difference between StratifiedKFold and StratifiedShuffleSplit in sklearn
  16. Save MinMaxScaler model in sklearn
  17. confusion matrix error “Classification metrics can’t handle a mix of multilabel-indicator and multiclass targets”
  18. Scikit-learn: How to obtain True Positive, True Negative, False Positive and False Negative
  19. Can anyone explain me StandardScaler?
  20. How to use sklearn fit_transform with pandas and return dataframe instead of numpy array?
  21. Mixing categorial and continuous data in Naive Bayes classifier using scikit-learn
  22. Tensorflow Precision / Recall / F1 score and Confusion matrix
  23. Got continuous is not supported error in RandomForestRegressor
  24. confused about random_state in decision tree of scikit learn
  25. How to compute jaccard similarity from a pandas dataframe
  26. Can I send callbacks to a KerasClassifier?
  27. How to get most informative features for scikit-learn classifier for different class?
  28. Model help using Scikit-learn when using GridSearch
  29. Determining the most contributing features for SVM classifier in sklearn
  30. scikit-learn .predict() default threshold

Leave a Comment