Trimming a huge (3.5 GB) csv file to read into R

by

My try with readLines. This piece of a code creates csv with selected years.

file_in <- file("in.csv","r")
file_out <- file("out.csv","a")
x <- readLines(file_in, n=1)
writeLines(x, file_out) # copy headers

B <- 300000 # depends how large is one pack
while(length(x)) {
    ind <- grep("^[^;]*;[^;]*; 20(09|10)", x)
    if (length(ind)) writeLines(x[ind], file_out)
    x <- readLines(file_in, n=B)
}
close(file_in)
close(file_out)

More Related Contents:

  1. How to read data when some numbers contain commas as thousand separator?
  2. What’s wrong with my function to load multiple .csv files into single dataframe in R using rbind?
  3. How to convert dataframe into time series?
  4. How to read in numbers with a comma as decimal separator?
  5. Convert comma separated string to numeric columns
  6. read.csv warning ‘EOF within quoted string’ prevents complete reading of file
  7. Specifying colClasses in the read.csv
  8. How do I import a CSV file in R? [closed]
  9. Merge 2 dataframes if value within range
  10. duplicate ‘row.names’ are not allowed error
  11. RStudio not picking the encoding I’m telling it to use when reading a file
  12. How to add a index by set of data when using rbindlist?
  13. Reading multiple csv files from a folder into a single dataframe in R [duplicate]
  14. Error in file(file, “rt”) : cannot open the connection [duplicate]
  15. Read and rbind multiple csv files
  16. How to check if CSV file has a comma or a semicolon as separator?
  17. R on Windows: character encoding hell
  18. Preserving large numbers
  19. How to get a .csv file into R?
  20. How to append a whole dataframe to a CSV in R
  21. Trying to merge multiple csv files in R
  22. Partially read really large csv.gz in R using vroom
  23. Save imported csv data in vector – R
  24. Calculate the mean of one column from several CSV files
  25. Reading a CSV file organized horizontally
  26. R: How to save lists into csv?
  27. consolidating data frames in R
  28. Importing “csv” file with multiple-character separator to R?
  29. How can I merge many data frames from csv files when the ID column is implied?
  30. What does the error “arguments imply differing number of rows: x, y” mean?

Leave a Comment