Custom transformer for sklearn Pipeline that alters both X and y

by

Modifying the sample axis, e.g. removing samples, does not (yet?) comply with the scikit-learn transformer API. So if you need to do this, you should do it outside any calls to scikit learn, as preprocessing.

As it is now, the transformer API is used to transform the features of a given sample into something new. This can implicitly contain information from other samples, but samples are never deleted.

Another option is to attempt to impute the missing values. But again, if you need to delete samples, treat it as preprocessing before using scikit learn.

More Related Contents:

  1. How to one-hot-encode from a pandas column containing a list?
  2. ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
  3. How to one hot encode variant length features?
  4. Scikit Learn OneHotEncoder fit and transform Error: ValueError: X has different shape than during fitting
  5. What are the pros and cons between get_dummies (Pandas) and OneHotEncoder (Scikit-learn)?
  6. TfidfVectorizer in scikit-learn : ValueError: np.nan is an invalid document
  7. How to use sklearn fit_transform with pandas and return dataframe instead of numpy array?
  8. Stratified Sampling in Pandas
  9. Replacing values represented by 'UN' with NaN
  10. pandas select from Dataframe using startswith
  11. What’s the fastest way in Python to calculate cosine similarity given sparse matrix data?
  12. Restart cumsum and get index if cumsum more than value
  13. How can I map True/False to 1/0 in a Pandas DataFrame?
  14. Dropping infinite values from dataframes in pandas?
  15. Is it possible to specify your own distance function using scikit-learn K-Means Clustering?
  16. ValueError: numpy.dtype has the wrong size, try recompiling
  17. Pandas: convert dtype ‘object’ to int
  18. Pandas lookup from one of multiple columns, based on value
  19. Most efficient way to sort an array into bins specified by an index array?
  20. Identifying statistical outliers with pandas: groupby and reduce rows into different dataframe
  21. What is dtype(‘O’), in pandas?
  22. pandas distinction between str and object types
  23. NumPy version of “Exponential weighted moving average”, equivalent to pandas.ewm().mean()
  24. Fastest pairwise distance metric in python
  25. Time Series Analysis – unevenly spaced measures – pandas + statsmodels
  26. Got continuous is not supported error in RandomForestRegressor
  27. confused about random_state in decision tree of scikit learn
  28. mean from pandas and numpy differ
  29. How to get most informative features for scikit-learn classifier for different class?
  30. scikit-learn .predict() default threshold

Leave a Comment