UTF8 to/from wide char conversion in STL

by

I’ve asked this question 5 years ago. This thread was very helpful for me back then, I came to a conclusion, then I moved on with my project. It is funny that I needed something similar recently, totally unrelated to that project from the past. As I was researching for possible solutions, I stumbled upon my own question 🙂

The solution I chose now is based on C++11. The boost libraries that Constantin mentions in his answer are now part of the standard. If we replace std::wstring with the new string type std::u16string, then the conversions will look like this:

UTF-8 to UTF-16

std::string source;
...
std::wstring_convert<std::codecvt_utf8_utf16<char16_t>,char16_t> convert;
std::u16string dest = convert.from_bytes(source);

UTF-16 to UTF-8

std::u16string source;
...
std::wstring_convert<std::codecvt_utf8_utf16<char16_t>,char16_t> convert;
std::string dest = convert.to_bytes(source);

As seen from the other answers, there are multiple approaches to the problem. That’s why I refrain from picking an accepted answer.

More Related Contents:

  1. does (w)ifstream support different encodings
  2. Inheriting and overriding functions of a std::string?
  3. How to convert wstring into string?
  4. What is the Windows equivalent for en_US.UTF-8 locale?
  5. Using Unicode in C++ source code
  6. How do you properly use WideCharToMultiByte
  7. Read Unicode UTF-8 file into wstring
  8. C programming: How to program for Unicode?
  9. C++ & Boost: encode/decode UTF-8
  10. Windows Unicode C++ Stream Output Failure
  11. Should I use wchar_t when using UTF-8?
  12. Do C++11 regular expressions work with UTF-8 strings?
  13. Specification of source charset encoding in MSVC++, like gcc “-finput-charset=CharSet”
  14. Erasing elements from a vector
  15. Reading a password from std::cin
  16. Is std::unique_ptr required to know the full definition of T?
  17. Why can I not push_back a unique_ptr into a vector?
  18. Is std::vector or boost::vector thread safe?
  19. remove_if equivalent for std::map
  20. Iteration over std::vector: unsigned vs signed index variable
  21. Using a STL map of function pointers
  22. How to read a growing text file in C++?
  23. Non-copyable elements in vector
  24. How to make elements of vector unique? (remove non adjacent duplicates)
  25. How does overloading of const and non-const functions work?
  26. Is a wide character string literal starting with L like L”Hello World” guaranteed to be encoded in Unicode?
  27. Does boost have a datatype for set operations that is simpler than the STL?
  28. Memory allocation and deallocation across dll boundaries
  29. STL deque accessing by index is O(1)?
  30. Can std::vector emplace_back copy construct from an element of the vector itself?

Leave a Comment