eval and quote in data.table

by

UPDATE (eddi): As of version 1.8.11 this has been fixed and .SD is not needed in cases where the expression can be evaluated in place, like in OP. Since currently presence of .SD triggers construction of full .SD, this will result in much faster speeds in some cases.

What’s going on is that calls to eval() are treated differently than you likely imagine in the code that implements [.data.table(). Specifically, [.data.table() contains special evaluation branches for i and j expressions that begin with the symbol eval. When you wrap the call to eval inside of a call to sum(), eval is no longer the first element of the parsed/substituted expression, and the special evaluation branch is skipped.

Here is the bit of code in the monster function displayed by typing getAnywhere("[.data.table") that makes a special allowance for calls to eval() passed in via [.data.table()‘s j-argument:

jsub = substitute(j)
    ...
    # Skipping some lines
    ...
jsubl = as.list.default(jsub)
if (identical(jsubl[[1L]], quote(eval))) {  # The test for eval 'on the outside' 
    jsub = eval(jsubl[[2L]], parent.frame(), parent.frame())
    if (is.expression(jsub)) 
        jsub = jsub[[1L]]
}

As a workaround, either follow the example in data.table FAQ 1.6 (pdf here), or explicitly point eval() towards .SD, the local variable that holds columns of whatever data.table you are operating on (here d). (For some more explanation of .SD‘s role, see the first few paragraphs of this answer).

d[, sum(eval(quoted_a, envir=.SD))]

More Related Contents:

  1. Naming columns in a data table in R
  2. How to create a lag variable within each group?
  3. data.table objects assigned with := from within function not printed
  4. What is the purpose of setting a key in data.table?
  5. .EACHI in data.table?
  6. Use a value from the previous row in an R data.table calculation
  7. How to efficiently calculate distance between pair of coordinates using data.table :=
  8. Sample random rows within each group in a data.table
  9. Melt using patterns when variable names contain string information – avoid coercion to numeric
  10. Error: package or namespace load failed for ggplot2 and for data.table
  11. How to perform join over date ranges using data.table?
  12. Split text string in a data.table columns
  13. How to speed up subset by groups
  14. knitr gets tricked by data.table `:=` assignment
  15. Add multiple columns to R data.table in one function call?
  16. Extract a column from a data.table as a vector, by position
  17. data.table equivalent of tidyr::complete()
  18. data.table merge based on date ranges
  19. Reason behind speed of fread in data.table package in R
  20. data.table row-wise sum, mean, min, max like dplyr?
  21. := (pass by reference) operator in the data.table package modifies another data table object simultaneously
  22. Rowwise cumulative sum
  23. Convert a data frame to a data.table without copy
  24. Row sums over columns with a certain pattern in their name
  25. data.table package in R 3.5 does not install
  26. Data Table – Select Value of Column by Name From Another Column
  27. paste two data.table columns
  28. Removing multiple columns from R data.table with parameter for columns to remove
  29. Merging multiple data.tables
  30. Fill option for fread

Leave a Comment