RStudio not picking the encoding I’m telling it to use when reading a file

by

This problem is caused by the wrong locale being set, whether inside RStudio or command-line R:

  1. If the problem only happens in RStudio not command-line R, go to RStudio->Preferences:General, tell us what ‘Default text encoding:’is set to, click ‘Change’ and try Windows-1252, UTF-8 or ISO8859-1(‘latin1’) (or else ‘Ask’ if you always want to be prompted). Screenshot attached at bottom. Let us know which one worked!

  2. If the problem also happens in command-line R, do the following:

Do locale -m on your Mac and tell us whether it supports CP1252 or else ISO8859-1 (‘latin1’)? Dump the list of supported locales if you need to. (You might as well tell us your version of MacOS while you’re at it.)

For both of those locales, try to change to that locale:

# first try Windows CP1252, although that's almost surely not supported on Mac:
Sys.setlocale("LC_ALL", "pt_PT.1252") # Make sure not to omit the `"LC_ALL",` first argument, it will fail.
Sys.setlocale("LC_ALL", "pt_PT.CP1252") # the name might need to be 'CP1252'

# next try IS08859-1(/'latin1'), this works for me:
Sys.setlocale("LC_ALL", "pt_PT.ISO8859-1")

# Try "pt_PT.UTF-8" too...

# in your program, make sure the Sys.setlocale worked, sprinkle this assertion in your code before attempting to read.csv:
stopifnot(Sys.getlocale('LC_CTYPE') == "pt_PT.ISO8859-1")

That should work.
Strictly the Sys.setlocale() command should go in your ~/.Rprofile for startup, not inside your R session or source-code.
However Sys.setlocale() can fail, so just be aware of that. Also, assert Sys.getlocale() inside your setup code early and often, as I do. (really, read.csv should figure out if the encoding it uses is compatible with the locale, and warn or error if not).

Let us know which fix worked! I’m trying to document this more generally so we can figure out the correct enhance.

  1. Screenshot of RStudio Preferences Change default text encoding menu:
    enter image description here

More Related Contents:

  1. R on Windows: character encoding hell
  2. How to source() .R file saved using UTF-8 encoding?
  3. Change R default library path using .libPaths in Rprofile.site fails to work
  4. What’s wrong with my function to load multiple .csv files into single dataframe in R using rbind?
  5. How to convert dataframe into time series?
  6. dynamically add plots to web page using shiny
  7. How to set up conda-installed R for use with RStudio?
  8. How to uninstall R and RStudio with all packages, settings and everything else?
  9. How to use objects from global environment in Rstudio Markdown
  10. How to write UTF-8 in a CSV file
  11. Merge 2 dataframes if value within range
  12. Set locale to system default UTF-8
  13. Delete “” from csv values and change column names when writing to a CSV
  14. duplicate ‘row.names’ are not allowed error
  15. How to add a index by set of data when using rbindlist?
  16. two-column layouts in RStudio presentations/slidify/pandoc
  17. R’s read.csv prepending 1st column name with junk text [duplicate]
  18. Get the path of current script
  19. add image in title page of rmarkdown pdf
  20. How to get unsaved script tabs
  21. Rstudio is duplicating commands in the command line
  22. Partially read really large csv.gz in R using vroom
  23. Save imported csv data in vector – R
  24. Scale and size of plot in RStudio shiny
  25. Sequence length encoding using R
  26. R: How to save lists into csv?
  27. consolidating data frames in R
  28. Importing “csv” file with multiple-character separator to R?
  29. How can I merge many data frames from csv files when the ID column is implied?
  30. Setting work directory in knitr using opts_chunk$set(root.dir = …) doesn’t work

Leave a Comment