Pandas: Chained assignments [duplicate]

by

The point of the SettingWithCopy is to warn the user that you may be doing something that will not update the original data frame as one might expect.

Here, data is a dataframe, possibly of a single dtype (or not). You are then taking a reference to this data['amount'] which is a Series, and updating it. This probably works in your case because you are returning the same dtype of data as existed.

However it could create a copy which updates a copy of data['amount'] which you would not see; Then you would be wondering why it is not updating.

Pandas returns a copy of an object in almost all method calls. The inplace operations are a convience operation which work, but in general are not clear that data is being modified and could potentially work on copies.

Much more clear to do this:

data['amount'] = data["amount"].fillna(data.groupby("num")["amount"].transform("mean"))

data["amount"] = data['amount'].fillna(mean_avg)

One further plus to working on copies. You can chain operations, this is not possible with inplace ones.

e.g.

data['amount'] = data['amount'].fillna(mean_avg)*2

And just an FYI. inplace operations are neither faster nor more memory efficient. my2c they should be banned. But too late on that API.

You can of course turn this off:

pd.set_option('chained_assignment',None)

Pandas runs with the entire test suite with this set to raise (so we know if chaining is happening) on, FYI.

More Related Contents:

  1. How to deal with SettingWithCopyWarning in Pandas
  2. How to add a new column to an existing DataFrame?
  3. why should I make a copy of a data frame in pandas
  4. What rules does Pandas use to generate a view vs a copy?
  5. Pandas: Subindexing dataframes: Copies vs views
  6. Pandas still getting SettingWithCopyWarning even after using .loc
  7. Action with pandas SettingWithCopyWarning
  8. Extracting specific selected columns to new DataFrame as a copy
  9. Pandas SettingWithCopyWarning [duplicate]
  10. Python: Pandas Dataframe how to multiply entire column with a scalar
  11. Checking whether data frame is copy or view in Pandas
  12. Pandas: Knowing when an operation affects the original dataframe
  13. Confusion about pandas copy of slice of dataframe warning
  14. How to iterate over rows in a DataFrame in Pandas
  15. How to copy files?
  16. extract column value based on another column pandas dataframe
  17. Pandas read nested json
  18. Plot Pandas DataFrame as Bar and Line on the same one chart
  19. Calculating difference between two rows in Python / Pandas
  20. Pandas: Looking up the list of sheets in an excel file
  21. Can pandas groupby aggregate into a list, rather than sum, mean, etc?
  22. Adding values for missing data combinations in Pandas
  23. python pandas – creating a column which keeps a running count of consecutive values
  24. Why is pandas.to_datetime slow for non standard time format such as ‘2014/12/31’
  25. What is the fastest and most efficient way to append rows to a DataFrame?
  26. How to plot different groups of data from a dataframe into a single figure
  27. how to concat two data frames with different column names in pandas? – python
  28. Pandas – Interleave / Zip two DataFrames by row
  29. how to specify the datetime format in read_csv
  30. Get total values_count from a dataframe with Python Pandas

Leave a Comment