jsonlite

ANOTHER UPDATE You can use the ndjson package to process this ndjson/streaming JSON data. It’s faster than jsonlite::stream_in() and always produces a completely “flat” data frame: system.time(bitly01 <- ndjson::stream_in(“usagov_bitly_data2013-05-17-1368832207.gz”)) ## user system elapsed ## 0.146 0.004 0.154 system.time(bitly02 <- jsonlite::stream_in(file(“usagov_bitly_data2013-05-17-1368832207.gz”), verbose=FALSE, pagesize=10000)) ## user system elapsed ## 0.419 0.008 0.427 If we examine the resultant … Read more

Update: 21 February 2016 col_fixer updated to include a vec2col argument that lets you flatten a list column into either a single string or a set of columns. In the data.frame you’ve downloaded, I see several different column types. There are normal columns comprising vectors of the same type. There are list columns where the … Read more