Read SAS sas7bdat data into R

by

sas7bdat worked fine for all but one of the files I was looking at (specifically, this one); in reporting the error to the sas7bdat developer, Matthew Shotwell, he also pointed me in the direction of Hadley’s haven package in R which also has a read_sas method.

This method is superior for two reasons:

1) It didn’t have any trouble reading the above-linked file
2) It is much (I’m talking much) faster than read.sas7bdat. Here’s a quick benchmark (on this file, which is smaller than the others) for evidence:

microbenchmark(times=10L,
               read.sas7bdat("psu97ai.sas7bdat"),
               read_sas("psu97ai.sas7bdat"))

Unit: milliseconds
                              expr        min         lq       mean     median         uq        max neval cld
 read.sas7bdat("psu97ai.sas7bdat") 66696.2955 67587.7061 71939.7025 68331.9600 77225.1979 82836.8152    10   b
      read_sas("psu97ai.sas7bdat")   397.9955   402.2627   410.4015   408.5038   418.1059   425.2762    10  a

That’s right–haven::read_sas takes (on average) 99.5% less time than sas7bdat::read.sas7bdat.

minor update

I previously wasn’t able to figure out whether the two methods produced the same data (i.e., that both have equal levels of fidelity with respect to reading the data), but have finally done so:

# Keep as data.tables
sas7bdat <- setDT(read.sas7bdat("psu97ai.sas7bdat"))
haven <- setDT(read_sas("psu97ai.sas7bdat"))

# read.sas7bdat prefers strings as factors,
#   and as of now has no stringsAsFactors argument
#   with which to prevent this
idj_factor <- sapply(haven, is.factor)

# Reset all factor columns as characters
sas7bdat[ , (idj_factor) := lapply(.SD, as.character), .SDcols = idj_factor]

# Check equality of the tables
all.equal(sas7bdat, haven, check.attributes = FALSE)
# [1] TRUE

However, note that read.sas7bdat has kept a massive list of attributes for the file, presumably a holdover from SAS:

str(sas7bdat)
# ...
# - attr(*, "column.info")=List of 70
#   ..$ :List of 12
#   .. ..$ name  : chr "NCESSCH"
#   .. ..$ offset: int 200
#   .. ..$ length: int 12
#   .. ..$ type  : chr "character"
#   .. ..$ format: chr "$"
#   .. ..$ fhdr  : int 0
#   .. ..$ foff  : int 76
#   .. ..$ flen  : int 1
#   .. ..$ label : chr "UNIQUE SCHOOL ID (NCES ASSIGNED)"
#   .. ..$ lhdr  : int 0
#   .. ..$ loff  : int 44
#   .. ..$ llen  : int 32
# ...

So, if by any chance you need these attributes (I know some people are particularly keen on the labels, for instance), perhaps read.sas7bdat is the option for you after all.

More Related Contents:

  1. Reshaping data.frame from wide to long format
  2. ggplot with 2 y axes on each side and different scales
  3. How to add leading zeros?
  4. Quickly reading very large tables as dataframes
  5. Order discrete x scale by frequency/value
  6. Combine two data frames by rows (rbind) when they have different sets of columns
  7. Counting unique / distinct values by group in a data frame
  8. Apply several summary functions on several variables by group in one call
  9. How to debug “contrasts can be applied only to factors with 2 or more levels” error?
  10. How to use a variable to specify column name in ggplot
  11. Formatting Decimal places in R
  12. How can I trim leading and trailing white space?
  13. Determine path of the executing script
  14. ggplot2 line chart gives “geom_path: Each group consist of only one observation. Do you need to adjust the group aesthetic?”
  15. How to name variables on the fly?
  16. Cleaning up factor levels (collapsing multiple levels/labels)
  17. Plot multiple columns on the same graph in R [duplicate]
  18. Concatenate a vector of strings/character
  19. How to generate permutations or combinations of object in R?
  20. Convert data.frame column format from character to factor
  21. Subset rows in a data frame based on a vector of values
  22. Convert UNIX epoch to Date object
  23. How to plot a function curve in R
  24. Use variable names in functions of dplyr
  25. Access variable value where the name of variable is stored in a string
  26. Remove groups with less than three unique observations
  27. List distinct values in a vector in R
  28. Display exact value of a variable in R
  29. What does “Error: object ” not found” mean?
  30. Is there anything wrong with using T & F instead of TRUE & FALSE?

Leave a Comment