Pandas Merge – How to avoid duplicating columns

by

You can work out the columns that are only in one DataFrame and use this to select a subset of columns in the merge.

cols_to_use = df2.columns.difference(df.columns)

Then perform the merge (note this is an index object but it has a handy tolist() method).

dfNew = merge(df, df2[cols_to_use], left_index=True, right_index=True, how='outer')

This will avoid any columns clashing in the merge.

More Related Contents:

  1. How to one-hot-encode from a pandas column containing a list?
  2. Rename Pandas DataFrame Index
  3. Pandas groupby cumulative sum
  4. Pandas DataFrame concat vs append
  5. Pandas counting and summing specific conditions
  6. Convert row to column header for Pandas DataFrame,
  7. Use AWS Glue Python with NumPy and Pandas Python Packages
  8. How to save the Pandas dataframe/series data as a figure?
  9. Apply CSS class to Pandas DataFrame using to_html
  10. matplotlib: plot multiple columns of pandas data frame on the bar chart
  11. Plotting categorical data with pandas and matplotlib
  12. Setting values on a copy of a slice from a DataFrame [duplicate]
  13. How can I replicate rows in Pandas?
  14. Pandas: SettingWithCopyWarning: [duplicate]
  15. Seaborn ValueError: zero-size array to reduction operation minimum which has no identity
  16. How to create a DataFrame of random integers with Pandas?
  17. summing two columns in a pandas dataframe
  18. add a row at top in pandas dataframe [duplicate]
  19. Find difference between two data frames
  20. Create new column based on values from other columns / apply a function of multiple columns, row-wise in Pandas
  21. Specifying date format when converting with pandas.to_datetime
  22. Bar graph from dataframe groupby
  23. What are the differences between feather and parquet?
  24. Grouped Bar graph Pandas
  25. KeyError when indexing Pandas dataframe
  26. pandas group by and assign a group id then ungroup
  27. How to iterate over columns of pandas dataframe to run regression
  28. Load CSV to Pandas MultiIndex DataFrame
  29. Python pandas – new column’s value if the item is in the list
  30. bash: Python import – Command not found for pandas

Leave a Comment