What is the difference between using loc and using just square brackets to filter for columns in Pandas/Python?

by

In the following situations, they behave the same:

  1. Selecting a single column (df['A'] is the same as df.loc[:, 'A'] -> selects column A)
  2. Selecting a list of columns (df[['A', 'B', 'C']] is the same as df.loc[:, ['A', 'B', 'C']] -> selects columns A, B and C)
  3. Slicing by rows (df[1:3] is the same as df.iloc[1:3] -> selects rows 1 and 2. Note, however, if you slice rows with loc, instead of iloc, you’ll get rows 1, 2 and 3 assuming you have a RangeIndex. See details here.)

However, [] does not work in the following situations:

  1. You can select a single row with df.loc[row_label]
  2. You can select a list of rows with df.loc[[row_label1, row_label2]]
  3. You can slice columns with df.loc[:, 'A':'C']

These three cannot be done with [].
More importantly, if your selection involves both rows and columns, then assignment becomes problematic.

df[1:3]['A'] = 5

This selects rows 1 and 2 then selects column ‘A’ of the returning object and assigns value 5 to it. The problem is, the returning object might be a copy so this may not change the actual DataFrame. This raises SettingWithCopyWarning. The correct way of making this assignment is:

df.loc[1:3, 'A'] = 5

With .loc, you are guaranteed to modify the original DataFrame. It also allows you to slice columns (df.loc[:, 'C':'F']), select a single row (df.loc[5]), and select a list of rows (df.loc[[1, 2, 5]]).

Also note that these two were not included in the API at the same time. .loc was added much later as a more powerful and explicit indexer. See unutbu’s answer for more detail.

Note: Getting columns with [] vs . is a completely different topic. . is only there for convenience. It only allows accessing columns whose names are valid Python identifiers (i.e. they cannot contain spaces, they cannot be composed of numbers…). It cannot be used when the names conflict with Series/DataFrame methods. It also cannot be used for non-existing columns (i.e. the assignment df.a = 1 won’t work if there is no column a). Other than that, . and [] are the same.

More Related Contents:

  1. How to drop rows of Pandas DataFrame whose value in a certain column is NaN
  2. Annotate bars with values on Pandas bar plots
  3. pandas: How do I split text in a column into multiple rows?
  4. Find row where values for column is maximal in a pandas DataFrame
  5. How to make separator in pandas read_csv more flexible wrt whitespace, for irregular separators?
  6. Why isn’t my Pandas ‘apply’ function referencing multiple columns working? [closed]
  7. Convert the string 2.90K to 2900 or 5.2M to 5200000 in pandas dataframe
  8. Conditionally fill column values based on another columns value in pandas
  9. python pandas extract year from datetime: df[‘year’] = df[‘date’].year is not working
  10. Pandas DataFrame stack multiple column values into single column
  11. How to replace negative numbers in Pandas Data Frame by zero
  12. Python Pandas Group by date using datetime data
  13. Removing the timestamp from a datetime in pandas dataframe
  14. Convert pandas data frame to series
  15. vectorize conditional assignment in pandas dataframe
  16. Pandas DataFrame merge summing column
  17. Binning a column with pandas
  18. Lambda including if…elif…else
  19. How can I replicate rows of a Pandas DataFrame?
  20. How to get/set a pandas index column title or name?
  21. Pandas DataFrame sort by categorical column but by specific class ordering
  22. Python pandas integer YYYYMMDD to datetime
  23. Pandas: drop columns with all NaN’s
  24. In Python pandas, start row index from 1 instead of zero without creating additional column
  25. Python: How to drop a row whose particular column is empty/NaN?
  26. Combine Pandas data frame column values into new column
  27. pandas extract year from datetime: df[‘year’] = df[‘date’].year is not working
  28. DataFrame object has no attribute append
  29. Python Pandas : pandas.to_datetime() is switching day & month when day is less than 13
  30. pandas.read_csv FileNotFoundError: File b’\xe2\x80\xaa’ despite correct path

Leave a Comment