How to convert DOS/Windows newline (CRLF) to Unix newline (LF) in a Bash script

by

You can use tr to convert from DOS to Unix; however, you can only do this safely if CR appears in your file only as the first byte of a CRLF byte pair. This is usually the case. You then use:

tr -d '\015' <DOS-file >UNIX-file

Note that the name DOS-file is different from the name UNIX-file; if you try to use the same name twice, you will end up with no data in the file.

You can’t do it the other way round (with standard ‘tr’).

If you know how to enter carriage return into a script (control-V, control-M to enter control-M), then:

sed 's/^M$//'     # DOS to Unix
sed 's/$/^M/'     # Unix to DOS

where the ‘^M’ is the control-M character. You can also use the bash ANSI-C Quoting mechanism to specify the carriage return:

sed $'s/\r$//'     # DOS to Unix
sed $'s/$/\r/'     # Unix to DOS

However, if you’re going to have to do this very often (more than once, roughly speaking), it is far more sensible to install the conversion programs (e.g. dos2unix and unix2dos, or perhaps dtou and utod) and use them.

If you need to process entire directories and subdirectories, you can use zip:

zip -r -ll zipfile.zip somedir/
unzip zipfile.zip

This will create a zip archive with line endings changed from CRLF to CR. unzip will then put the converted files back in place (and ask you file by file – you can answer: Yes-to-all). Credits to @vmsnomad for pointing this out.

More Related Contents:

  1. How to convert DOS/Windows newline (CRLF) to Unix newline (LF)
  2. Looping through the content of a file in Bash
  3. How do SO_REUSEADDR and SO_REUSEPORT differ?
  4. Why should eval be avoided in Bash, and what should I use instead?
  5. How to permanently set $PATH on Linux/Unix [closed]
  6. Setting an argument with bash [duplicate]
  7. What’s the magic of “-” (a dash) in command-line parameters?
  8. How to generate a core dump in Linux on a segmentation fault?
  9. How to get overall CPU usage (e.g. 57%) on Linux [closed]
  10. Aborting a shell script if any command returns a non-zero value
  11. How to merge two files using AWK? [duplicate]
  12. How to set environment variable for everyone under my linux system?
  13. What is the difference between using `sh` and `source`?
  14. Setting environment variables in Linux using Bash
  15. What does `set -x` do?
  16. Does “argument list too long” restriction apply to shell builtins?
  17. Given two directory trees, how can I find out which files differ by content? [closed]
  18. How to sort a file in-place?
  19. How do I syntax check a Bash script without running it?
  20. Is it OK to use the same input file as output of a piped command?
  21. Uppercasing First Letter of Words Using SED
  22. How to sort strings that contain a common prefix and suffix numerically from Bash?
  23. Why can’t I use Unix Nohup with Bash For-loop?
  24. When to wrap quotes around a shell variable?
  25. How do you normalize a file path in Bash?
  26. How do I get cURL to not show the progress bar?
  27. Using `date` command to get previous, current and next month
  28. Joining multiple fields in text files on Unix
  29. How to check if a program is run in Bash on Ubuntu on Windows and not just plain Ubuntu?
  30. What are the rules for valid identifiers (e.g. functions, vars, etc) in Bash?

Leave a Comment