How should I use g++’s -finput-charset compiler option correctly in order to compile a non-UTF-8 source file?

Encoding Blues

You cannot use UTF-16 for source code files; because the header you are including, <iostream>, is not UTF-16-encoded. As #include includes the files verbatim, this means that you suddenly have an UTF-16-encoded file with a large chunk (approximately 4k, apparently) of invalid data.

There is almost no good reason to ever use UTF-16 for anything, so this is just as well.

Regarding problems with encoding support: The OSes themselves are not responsible for providing encoding support; this comes down to the compilers used.

g++ on Windows supports absolutely all of the same encodings as g++ on Linux, because it’s the same program, unless whatever version of g++ you are using on Windows relies on a deeply broken iconv library.

Inspect your toolchain and ensure that all your tools are in working order.

As an alternative; don’t use Chinese in the source files, but write them in English, using English-language literals, or simple TOKEN_STYLE_PLACEHOLDERs, using l10n and i18n to replace these in the running executable.

-finput-charset is almost certainly a holdover from the days of code pages and other nonsense of the kind; however; an ISO-8859-n file will almost always be compatible with UTF-8 standard headers.

For the next time; remember a simple mantra: “N’DUUH!”; “Never Don’t Use UTF-8!”


I18N

A common solution to this kind of problem is to remove the problem entirely, by way of, for instance, gettext.

When using gettext, you usually end up with a function loc(char *) that abstracts away most of the translation tool specific code. So, instead of

#include <iostream>

int main () {
  std::cout << "瓜田李下" << std::endl;
}

You would have

#include <iostream>

#include "translation.h"

int main () {
  std::cout << loc("DEEPER_MEANING") << std::endl;
}

and, in zh.po:

msgid DEEPER_MEANING
msgstr "瓜田李下"

Of course, you could also then have a en.po:

msgid DEEPER_MEANING
msgstr "Still waters run deep"

This can be expanded upon, and the gettext package has tools for expansion of strings with variables and such, or you could use printf, to account for different grammars.


The Third Option

Instead of having to deal with multiple compilers with different requirements for file encodings, file endings, byte order marks, and other problems of the kind; it is possible to cross-compile using MinGW or similar tools.

This option requires some setup, but it may very well reduce future overhead and headaches.

Leave a Comment