Replace duplicate values across columns in Pandas

by

You can use the duplicated method to return a boolean indexer of whether elements are duplicates or not:

In [214]: pd.Series(['M', '0', 'M', '0']).duplicated()
Out[214]:
0    False
1    False
2     True
3     True
dtype: bool

Then you could create a mask by mapping this across the rows of your dataframe, and using where to perform your substitution:

is_duplicate = df.apply(pd.Series.duplicated, axis=1)
df.where(~is_duplicate, 0)

  col1 col2 col3 col4
0    A    B    C    0
1    M    0    0    0
2    B    0    0    0
3    X    0    Y    0

More Related Contents:

  1. Selecting with complex criteria from pandas.DataFrame
  2. Combine two pandas Data Frames (join on a common column)
  3. Restart cumsum and get index if cumsum more than value
  4. Remove non-numeric rows in one column with pandas
  5. How to deal with multi-level column names downloaded with yfinance
  6. Replace all occurrences of a string in a pandas dataframe (Python)
  7. Running get_dummies on several DataFrame columns?
  8. ValueError: numpy.dtype has the wrong size, try recompiling
  9. Get business days between start and end date using pandas
  10. How to change the datetime tick label frequency for matplotlib plots
  11. Pandas DataFrame aggregate function using multiple columns
  12. Subtract two columns in dataframe
  13. Python Pandas: Is Order Preserved When Using groupby() and agg()?
  14. Pass percentiles to pandas agg function
  15. How to calculate time difference by group using pandas?
  16. Capitalize first letter of each word in a dataframe column
  17. Compute the running (cumulative) maximum for a series in pandas
  18. Get total of Pandas column
  19. Sorting columns and selecting top n rows in each group pandas dataframe
  20. Get first and second highest values in pandas columns
  21. ImportError: No module named ‘pandas.indexes’
  22. select columns based on columns names containing a specific string in pandas
  23. python dask DataFrame, support for (trivially parallelizable) row apply?
  24. Enumerate each row for each group in a DataFrame
  25. Python How to use ExcelWriter to write into an existing worksheet
  26. Pandas combining rows based on dates
  27. Collapsing rows in a Pandas dataframe if all rows have only one value in their columns
  28. Preserving column order in Python Pandas DataFrame
  29. Convert dates to pd.to_datetime where month could be either a number or month name
  30. Multi Index Sorting in Pandas

Leave a Comment