How to remove non UTF-8 characters from text file

by

This command:

iconv -f utf-8 -t utf-8 -c file.txt

will clean up your UTF-8 file, skipping all the invalid characters.

-f is the source format
-t the target format
-c skips any invalid sequence

More Related Contents:

  1. how to detect invalid utf8 unicode/binary in a text file
  2. create a file with prefix =file1 followed by the random number
  3. How to read a file line by line and making a new file for each line having same filename as last word of each line? [closed]
  4. Why to code a script which will check if some package is installed in linux and if not then install it?
  5. Read values into a shell variable from a pipe
  6. Aborting a shell script if any command returns a non-zero value
  7. Fastest way to find lines of a file from another larger file in Bash
  8. Use sudo with password as parameter [closed]
  9. Shell script: Run function from script over ssh
  10. What is the difference between using `sh` and `source`?
  11. Setting environment variables in Linux using Bash
  12. Get the Perl rename utility instead of the built-in rename
  13. Automating telnet session using bash scripts
  14. Run a shell command when a file is added
  15. How do I syntax check a Bash script without running it?
  16. how to get MouseMove and MouseClick in bash?
  17. Find the files that have been changed in last 24 hours
  18. Tar a directory, but don’t store full absolute paths in the archive
  19. Bash script execution with and without shebang in Linux and BSD
  20. Variable in Bash Script that keeps it value from the last time running
  21. bash command not found when setting a variable
  22. How to obtain the number of CPUs/cores in Linux from the command line?
  23. How do I get cURL to not show the progress bar?
  24. Move all files except one
  25. Linux: remove file extensions for multiple files
  26. Joining multiple fields in text files on Unix
  27. Run script with rc.local: script works, but not at boot
  28. Maximum number of Bash arguments != max num cp arguments?
  29. Write a bash shell script that consumes a constant amount of RAM for a user defined time [closed]
  30. Using “$RANDOM” to generate a random string in Bash

Leave a Comment