Python pandas rolling_apply two column input into function

by

Not sure if still relevant here, with the new rolling classes on pandas, whenever we pass raw=False to apply, we are actually passing the series to the wraper, which means we have access to the index of each observation, and can use that to further handle multiple columns.

From the docs:

raw : bool, default None

False : passes each row or column as a Series to the function.

True or None : the passed function will receive ndarray objects instead. If you are just applying a NumPy reduction function this will achieve much better performance.

In this scenario, we can do the following:

### create a func for multiple columns
def cust_func(s):

    val_for_col2 = df.loc[s.index, col2] #.values
    val_for_col3 = df.loc[s.index, col3] #.values
    val_for_col4 = df.loc[s.index, col4] #.values
    
    ## apply over multiple column values
    return np.max(s) *np.min(val_for_col2)*np.max(val_for_col3)*np.mean(val_for_col4)
    

### Apply to the dataframe
df.rolling('10s')['col1'].apply(cust_func, raw=False)

Note that here we can still use all functionalities from pandas rolling class, which is particularly useful when dealing with time-related windows.

The fact that we are passing one column and using the entire dataframe feels like a hack, but it works in practice.

More Related Contents:

  1. Clustered Stacked Bar in Python Pandas [closed]
  2. How to add a new column to an existing DataFrame?
  3. Python Pandas – Find difference between two data frames
  4. Convert pandas timezone-aware DateTimeIndex to naive timestamp, but in certain timezone
  5. Pandas groupby cumulative sum
  6. Pandas DataFrame concat vs append
  7. Use pandas.shift() within a group
  8. What is the most efficient way of counting occurrences in pandas?
  9. Pandas read csv file with float values results in weird rounding and decimal digits
  10. pandas – filter dataframe by another dataframe by row elements
  11. Pandas interpolate within a groupby
  12. Efficiently replace values from a column to another column Pandas DataFrame
  13. Python Pandas: How to read only first n rows of CSV files in?
  14. Pandas – replacing column values
  15. add columns different length pandas
  16. Pandas get the age from a date (example: date of birth)
  17. How to create a grouped bar plot
  18. Sample each group after pandas groupby
  19. Python Pandas – Missing required dependencies [‘numpy’] 1
  20. Pandas groupby for zero values
  21. Plot multiple Y axes
  22. Pandas Data Frame how to merge columns
  23. matplotlib plot window won’t appear
  24. Python – Convert datetime column into seconds [duplicate]
  25. How to display percentage above grouped bar chart
  26. Reverse DataFrame column order
  27. Using regex matched groups in pandas dataframe replace function
  28. converting list of header and row lists into pandas DataFrame
  29. How to return a csv file/Pandas DataFrame in JSON format using FastAPI?
  30. How to get number of groups in a groupby object in pandas?

Leave a Comment