how do I remove rows with duplicate values of columns in pandas data frame?

by

Using drop_duplicates with subset with list of columns to check for duplicates on and keep='first' to keep first of duplicates.

If dataframe is:

df = pd.DataFrame({'Column1': ["'cat'", "'toy'", "'cat'"],
                   'Column2': ["'bat'", "'flower'", "'bat'"],
                   'Column3': ["'xyz'", "'abc'", "'lmn'"]})
print(df)

Result:

  Column1   Column2 Column3
0   'cat'     'bat'   'xyz'
1   'toy'  'flower'   'abc'
2   'cat'     'bat'   'lmn'

Then:

result_df = df.drop_duplicates(subset=['Column1', 'Column2'], keep='first')
print(result_df)

Result:

  Column1   Column2 Column3
0   'cat'     'bat'   'xyz'
1   'toy'  'flower'   'abc'

More Related Contents:

  1. Apply multiple functions to multiple groupby columns
  2. Pandas percentage of total with groupby
  3. Scatter plots in Pandas/Pyplot: How to plot by category [duplicate]
  4. Using Pandas to pd.read_excel() for multiple worksheets of the same workbook
  5. How to add pandas data to an existing csv file?
  6. How to explode a list inside a Dataframe cell into separate rows
  7. How do I operate on a DataFrame with a Series for every column?
  8. Improve subplot size/spacing with many subplots in matplotlib
  9. Pandas split column into multiple columns by comma
  10. Repeat rows in a pandas DataFrame based on column value
  11. Accessing every 1st element of Pandas DataFrame column containing lists
  12. How to use groupby to concatenate strings in python pandas?
  13. Horizontal stacked bar plot and add labels to each section
  14. How to flatten a nested JSON recursively, with flatten_json
  15. Python – How to convert JSON File to Dataframe
  16. How to invoke pandas.rolling.apply with parameters from multiple column?
  17. How to remove timezone from a Timestamp column in a pandas dataframe
  18. Pandas expand rows from list data available in column
  19. Splitting timestamp column into separate date and time columns
  20. Create Column with ELIF in Pandas
  21. ‘utf-8’ codec can’t decode byte 0x92 in position 18: invalid start byte
  22. Pandas – Compute z-score for all columns
  23. How can I use cumsum within a group in Pandas?
  24. Find date range overlap in python
  25. How to get unique values from multiple columns in a pandas groupby
  26. Merge a list of pandas dataframes
  27. How do I delete a column that contains only zeros in Pandas?
  28. How to repeat a Pandas DataFrame?
  29. How to efficiently assign unique ID to individuals with multiple entries based on name in very large df
  30. Add image annotations to bar plots

Leave a Comment