Subset rows corresponding to max value by group using data.table

by

Here’s the fast data.table way:

bdt[bdt[, .I[g == max(g)], by = id]$V1]

This avoids constructing .SD, which is the bottleneck in your expressions.

edit: Actually, the main reason the OP is slow is not just that it has .SD in it, but the fact that it uses it in a particular way – by calling [.data.table, which at the moment has a huge overhead, so running it in a loop (when one does a by) accumulates a very large penalty.

More Related Contents:

  1. Understanding exactly when a data.table is a reference to (vs a copy of) another data.table
  2. Apply a function to every specified column in a data.table and update by reference
  3. dcast warning: ‘Aggregation function missing: defaulting to length’
  4. Fastest way to replace NAs in a large data.table
  5. Fastest way to add rows for missing time steps?
  6. Summarizing multiple columns with data.table
  7. Why is rbindlist “better” than rbind?
  8. Aggregate a dataframe on a given column and display another column
  9. Apply multiple functions to multiple columns in data.table
  10. data.table join then add columns to existing data.frame without re-copy
  11. Convert column classes in data.table
  12. Using lists inside data.table columns
  13. Perform a semi-join with data.table
  14. Find the intersection of overlapping ranges in two tables using data.table function foverlaps
  15. Adding new columns to a data.table by-reference within a function not always working
  16. Referring to data.table columns by names saved in variables
  17. Calculate multiple aggregations on several variables using lapply(.SD, …)
  18. Merge rows in a dataframe where the rows are disjoint and contain NAs
  19. Calculate row-wise maximum
  20. Efficient row-wise operations on a data.table
  21. Replace logical values (TRUE / FALSE) with numeric (1 / 0)
  22. How to change type of target column when doing := by group in a data.table in R?
  23. Insert a row in a data.table
  24. rolling joins data.table in R
  25. Efficiently merging two data frames on a non-trivial criteria
  26. How to select R data.table rows based on substring match (a la SQL like)
  27. Any way to force fread() of data.table not to stop on empty lines?
  28. R data table: update join
  29. Split a string column into several dummy variables
  30. R fast single item lookup from list vs data.table vs hash

Leave a Comment