sklearn pipeline – how to apply different transformations on different columns

by

The way I usually do it is with a FeatureUnion, using a FunctionTransformer to pull out the relevant columns.

Important notes:

  • You have to define your functions with def since annoyingly you can’t use lambda or partial in FunctionTransformer if you want to pickle your model

  • You need to initialize FunctionTransformer with validate=False

Something like this:

from sklearn.pipeline import make_union, make_pipeline
from sklearn.preprocessing import FunctionTransformer

def get_text_cols(df):
    return df[['name', 'fruit']]

def get_num_cols(df):
    return df[['height','age']]

vec = make_union(*[
    make_pipeline(FunctionTransformer(get_text_cols, validate=False), LabelEncoder()))),
    make_pipeline(FunctionTransformer(get_num_cols, validate=False), MinMaxScaler())))
])

More Related Contents:

  1. Getting model attributes from pipeline
  2. Sklearn Pipeline: Get feature names after OneHotEncode In ColumnTransformer
  3. python kmeans on string
  4. Why does the following code cant get value of n_fold = 1?
  5. How to extract the decision rules from scikit-learn decision-tree?
  6. How to one-hot-encode from a pandas column containing a list?
  7. scikit-learn & statsmodels – which R-squared is correct?
  8. ImportError in importing from sklearn: cannot import name check_build
  9. Random state (Pseudo-random number) in Scikit learn
  10. Error importing scikit-learn modules
  11. How to one hot encode variant length features?
  12. pandas dataframe columns scaling with sklearn
  13. Clustering text documents using scikit-learn kmeans in Python
  14. scikit-learn cross validation, negative values with mean squared error
  15. Efficiently create sparse pivot tables in pandas?
  16. ROC for multiclass classification
  17. Passing categorical data to Sklearn Decision Tree
  18. fit-transform on training data and transform on test data [duplicate]
  19. Scikit Learn OneHotEncoder fit and transform Error: ValueError: X has different shape than during fitting
  20. Efficiently count word frequencies in python
  21. How to find the importance of the features for a logistic regression model?
  22. Scikit Learn GridSearchCV without cross validation (unsupervised learning)
  23. Mixing categorial and continuous data in Naive Bayes classifier using scikit-learn
  24. sklearn : TFIDF Transformer : How to get tf-idf values of given words in document
  25. Using scikit to determine contributions of each feature to a specific class prediction
  26. In sklearn.decomposition.PCA, why are components_ negative?
  27. Pyinstaller ; ModuleNotFoundError: No module named ‘sklearn.utils._cython_blas’
  28. Model help using Scikit-learn when using GridSearch
  29. Does the SVM in sklearn support incremental (online) learning?
  30. Determining the most contributing features for SVM classifier in sklearn

Leave a Comment