How can I use std::imbue to set the locale for std::wcout?

by

First I did some more test using your code and I can confirm that L"Преступление и наказание" is a correct UTF16 string. I controlled the code of the individual characters, and they are correctly 0x41f, 0x440, 0x435, 0x441, 0x442, 0x443, 0x43f, 0x43b, 0x435, 0x43d, 0x438, 0x435, 0x20, 0x438, 0x20, 0x43d, 0x430, 0x43a, 0x430, 0x437, 0x430, 0x43d, 0x438, 0x435

I could not find any reference about it, but it looks like simply calling imbue is not enough. imbue it a method from basic_ios which is an ancestor of cout and wcout. It does act on numeric conversions, but on all my tests, it has no effect on the charset used for output.

By default, the locale used in a C++ (or C) program is … the C locale which knows nothing about unicode. All printable ASCII characters (below 128) are outputted as is, and others are replaced with a ?. It is exactly what your program does.

To make it work correctly, you have to select a locale that knows about unicode characters with setlocale. Once this is done, you can change the numeric conversion by calling imbue, and as you selected a unicode charset all will be fine.

So provided your current locale uses an UTF-8 charset, you only have to add

setlocale(LC_ALL, "");

as first line in your program, and the output will be as expected :

0: "Преступление"
1: "и"
2: "наказание"

I counted 3 words.
and the last word was "наказание"

If your current locale does not use UTF-8, choose one that is installed on you system and that supports it. I used setlocale(LC_ALL, "fr_FR.UTF-8");, or even setlocale(LC_ALL, "en_US.UTF-8"); and both worked.

Edit :

In fact, the best way to correctly output unicode to screen is to use setlocale(LC_ALL, "");. It automatically adapts to the current charset. I tested with a stripped down variant using Latin1 charset (my system speaks natively french and not russian …)

#include <iostream>
#include <locale>

using namespace std;

int main() {
    setlocale(LC_ALL, "");
    wchar_t ws[] = { 0xe8, 0xe9, 0 };

    wcout << ws << endl;
}

I tried it under Linux using UTF-8 charset and ISO-8859-1 (latin1) (resp export LANG=fr_FR.UTF-8 and export LANG=fr_FR.ISO-8859-1) and I got correctly èé in the proper charset. I tried it also under Windows XP, with codepage 851 (oem) and 1252 (ansi) (resp. chcp 850 and chcp 1252 with Lucida console charset), and got èé on the console too.

Edit 2 :

Of course, you can also set a global C++ locale with locale::global(locale(""); with default locale or locale::global(locale("ru_RU.UTF-8"); with russian locale, but it is more than simply calling setlocale. According to the documentation of Gnu implementation of C++ Standard Library about locale : there is only one relation (of the C++ locale mechanism) to the C locale mechanism: the global C locale is modified if a named C++ locale object is set as the global locale”, that is: std::locale::global(std::locale("")); affects the C functions as if the following call was made: std::setlocale(LC_ALL, "");. On the other hand, there is no vice versa, that is, calling setlocale has no whatsoever on the C++ locale mechanism, in particular on the working of locale(“”).

So it really looks like there was an underlying C library mechanizme that should be first enabled with setlocale to allow imbue conversion to work correctly.

More Related Contents:

  1. Is it possible to print a variable’s type in standard C++?
  2. What is std::move(), and when should it be used?
  3. How does #include work in C++? [duplicate]
  4. std::function vs template
  5. Why is the destructor of a future returned from `std::async` blocking?
  6. When is a private constructor not a private constructor?
  7. C++11 “auto” semantics
  8. Reading from ifstream won’t read whitespace
  9. Is there a non-atomic equivalent of std::shared_ptr? And why isn’t there one in ?
  10. When is overloading pass by reference (l-value and r-value) preferred to pass-by-value?
  11. Does the GotW #101 “solution” actually solve anything?
  12. Lambda capture and parameter with same name – who shadows the other? (clang vs gcc)
  13. Is there any compiler barrier which is equal to asm(“” ::: “memory”) in C++11?
  14. What are unevaluated contexts in C++?
  15. Configuring the GCC compiler switches in Qt, QtCreator, and QMake
  16. How to do serialization of Class having members of custom data types in C++? [closed]
  17. Does std::mt19937 require warmup?
  18. What’s the difference between parentheses and braces in c++ when constructing objects
  19. cc1plus: error: unrecognized command line option “-std=c++11” with g++
  20. How do I pass multiple ints into a vector at once?
  21. Is it wise to ignore gcc/clang’s “-Wmissing-braces” warning?
  22. Why does pointer to int convert to void* but pointer to function convert to bool?
  23. Can std::vector emplace_back copy construct from an element of the vector itself?
  24. boost::asio + std::future – Access violation after closing socket
  25. Should all/most setter functions in C++11 be written as function templates accepting universal references?
  26. Is codecvt not a std header?
  27. std::thread::join() hangs if called after main() exits when using VS2012 RC
  28. Is there a way to statically-initialize a dynamically-allocated array in C++?
  29. How to get duration, as int milli’s and float seconds from ?
  30. get part of std::tuple

Leave a Comment