Why is as.Date slow on a character vector?

by

As others mentioned, strptime (converting from character to POSIXlt) is the bottleneck here. Another simple solution uses the lubridate package and its fast_strptime method instead.

Here’s what it looks like on my data:

> tables()
     NAME      NROW  MB COLS                                     
[1,] pp   3,718,339 126 session_id,date,user_id,path,num_sessions
     KEY         
[1,] user_id,date
Total: 126MB

> pp[, 2]
               date
      1: 2013-09-25
      2: 2013-09-25
      3: 2013-09-25
      4: 2013-09-25
      5: 2013-09-25
     ---           
3718335: 2013-09-25
3718336: 2013-09-25
3718337: 2013-09-25
3718338: 2013-10-11
3718339: 2013-10-11

> system.time(pp[, date := as.Date(fast_strptime(date, "%Y-%m-%d"))])
   user  system elapsed 
  0.315   0.026   0.344

For comparison:

> system.time(pp[, date := as.Date(date, "%Y-%m-%d")])
   user  system elapsed 
108.193   0.399 108.844

That’s ~316 times faster!

More Related Contents:

  1. Understanding exactly when a data.table is a reference to (vs a copy of) another data.table
  2. Subset rows corresponding to max value by group using data.table
  3. Extract row corresponding to minimum value of a variable by group
  4. Group by multiple columns and sum other multiple columns
  5. Apply a function to every specified column in a data.table and update by reference
  6. Fastest way to replace NAs in a large data.table
  7. Left join using data.table
  8. How can one work fully generically in data.table in R with column names in variables
  9. Faster version of combn
  10. Pass column name in data.table using variable [duplicate]
  11. What does .SD stand for in data.table in R
  12. data.table join then add columns to existing data.frame without re-copy
  13. data.table – select first n rows within group [duplicate]
  14. subsetting a data.table using != excludes NA too
  15. Adding new columns to a data.table by-reference within a function not always working
  16. Dynamic column names in data.table
  17. Calculate multiple aggregations on several variables using lapply(.SD, …)
  18. Convert *some* column classes in data.table
  19. from data table, randomly select one row per group
  20. How to change type of target column when doing := by group in a data.table in R?
  21. Long Numbers As A Character String
  22. Insert a row in a data.table
  23. rolling joins data.table in R
  24. Efficiently merging two data frames on a non-trivial criteria
  25. How to select R data.table rows based on substring match (a la SQL like)
  26. Any way to force fread() of data.table not to stop on empty lines?
  27. R data table: update join
  28. Split a string column into several dummy variables
  29. Is there a _fast_ way to run a rolling regression inside data.table?
  30. R fast single item lookup from list vs data.table vs hash

Leave a Comment