How can I clean source code files of invisible characters?

by

You don’t get the character in the editor, because you can’t find it in text editors. #FEFF or #FFFE are so-called byte-order marks. They are a Microsoft invention to tell in a Unicode file, in which order multi-byte characters are stored.

To get rid of it, tell your editor to save the file either as ANSI/ISO-8859 or as Unicode without BOM. If your editor can’t do so, you’ll either have to switch editors (sadly) or use some kind of truncation tool like, e.g., a hex editor that allows you to see how the file really looks.

On googling, it seems, that TextWrangler has a “UTF-8, no BOM” mode. Otherwise, if you’re comfortable with the terminal, you can use Vim:

:set nobomb

and save the file. Presto!

The characters are always the very first in a text file. Editors with support for the BOM will not, as I mentioned, show it to you at all.

More Related Contents:

  1. Java char output isn’t consistent
  2. What’s the difference between UTF-8 and UTF-8 without BOM?
  3. What’s the complete range for Chinese characters in Unicode?
  4. What is the difference between UTF-8 and Unicode?
  5. What are Unicode, UTF-8, and UTF-16?
  6. Using awk to remove the Byte-order mark
  7. What is the proper way to URL encode Unicode characters?
  8. FPDF utf-8 encoding (HOW-TO)
  9. ‘Fill’ Unicode characters in labels
  10. What’s the difference between Unicode and UTF-8? [duplicate]
  11. Really Good, Bad UTF-8 example test data [closed]
  12. How does UTF-8 “variable-width encoding” work?
  13. Creating Unicode character from its number
  14. What are the most common non-BMP Unicode characters in actual use? [closed]
  15. Unicode Identifiers and Source Code in C++11?
  16. Invisible characters – ASCII
  17. Should I use accented characters in URLs?
  18. How can I get the Unicode code point(s) of a Character?
  19. python 3.0, how to make print() output unicode?
  20. What is the range of Unicode Printable Characters?
  21. complete, monospaced Unicode font? [closed]
  22. Is there a Unicode glyph that looks like a “key” icon? [closed]
  23. Convert unicode code point to literal character in Go
  24. Using box-drawing Unicode characters in batch files
  25. UTF-8 & Unicode, what’s with 0xC0 and 0x80?
  26. What does u’\ufe0f’ in an emoji mean? Is it the same if I delete it?
  27. How can I use Unicode characters on the Windows command line?
  28. Issue with smtplib sending mail with unicode characters in Python 3.1
  29. Input unicode string with pyautogui
  30. unicode().decode(‘utf-8’, ‘ignore’) raising UnicodeEncodeError

Leave a Comment