What does .SD stand for in data.table in R

by

.SD stands for something like “Subset of Data.table”. There’s no significance to the initial ".", except that it makes it even more unlikely that there will be a clash with a user-defined column name.

If this is your data.table:

DT = data.table(x=rep(c("a","b","c"),each=2), y=c(1,3), v=1:6)
setkey(DT, y)
DT
#    x y v
# 1: a 1 1
# 2: b 1 3
# 3: c 1 5
# 4: a 3 2
# 5: b 3 4
# 6: c 3 6

Doing this may help you see what .SD is:

DT[ , .SD[ , paste(x, v, sep="", collapse="_")], by=y]
#    y       V1
# 1: 1 a1_b3_c5
# 2: 3 a2_b4_c6

Basically, the by=y statement breaks the original data.table into these two sub-data.tables

DT[ , print(.SD), by=y]
# <1st sub-data.table, called '.SD' while it's being operated on>
#    x v
# 1: a 1
# 2: b 3
# 3: c 5
# <2nd sub-data.table, ALSO called '.SD' while it's being operated on>
#    x v
# 1: a 2
# 2: b 4
# 3: c 6
# <final output, since print() doesn't return anything>
# Empty data.table (0 rows) of 1 col: y

and operates on them in turn.

While it is operating on either one, it lets you refer to the current sub-data.table by using the nick-name/handle/symbol .SD. That’s very handy, as you can access and operate on the columns just as if you were sitting at the command line working with a single data.table called .SD … except that here, data.table will carry out those operations on every single sub-data.table defined by combinations of the key, “pasting” them back together and returning the results in a single data.table!

More Related Contents:

  1. Is there a dplyr equivalent to data.table::rleid?
  2. Assign multiple columns using := in data.table, by group
  3. Efficient way to rbind data.frames with different columns
  4. How to replace NA values in a table for selected columns
  5. Select multiple columns in data.table by their numeric indices
  6. Adaptive moving average – top performance in R
  7. Finding Overlaps between interval sets / Efficient Overlap Joins
  8. Force character vector encoding from “unknown” to “UTF-8” in R
  9. Operator == inconsistent in logical columns in data.table
  10. Rolling join on data.table with duplicate keys
  11. Why is using update on a lm inside a grouped data.table losing its model data?
  12. Complex non-equi merge in R
  13. eval and quote in data.table
  14. Efficiently computing a linear combination of data.table columns
  15. When should I use the := operator in data.table?
  16. Subsetting data.table by 2nd column only of a 2 column key, using binary search not vector scan
  17. Using dynamic column names in `data.table`
  18. Conditional binary join and update by reference using the data.table package
  19. How do you delete a column by name in data.table?
  20. How does one change the levels of a factor column in a data.table
  21. When should I use setDT() instead of data.table() to create a data.table?
  22. DT[!(x == .)] and DT[x != .] treat NA in x inconsistently
  23. In R data.table, how do I pass variable parameters to an expression?
  24. Merging two sets of data by data.table roll=’nearest’ function
  25. Using data.table i and j arguments in functions
  26. R data.table breaks in exported functions
  27. Empty factors in “by” data.table
  28. How to use data.table within functions and loops?
  29. What you can do with a data.frame that you can’t with a data.table?
  30. data.table: anonymous function in j

Leave a Comment