Do C++11 regular expressions work with UTF-8 strings?

by

You would need to test your compiler and the system you are using, but in theory, it will be supported if your system has a UTF-8 locale. The following test returned true for me on Clang/OS X.

bool test_unicode()
{
    std::locale old;
    std::locale::global(std::locale("en_US.UTF-8"));

    std::regex pattern("[[:alpha:]]+", std::regex_constants::extended);
    bool result = std::regex_match(std::string("abcdéfg"), pattern);

    std::locale::global(old);

    return result;
}

NOTE: This was compiled in a file what was UTF-8 encoded.

Just to be safe I also used a string with the explicit hex versions. It worked also.

bool test_unicode2()
{
    std::locale old;
    std::locale::global(std::locale("en_US.UTF-8"));

    std::regex pattern("[[:alpha:]]+", std::regex_constants::extended);
    bool result = std::regex_match(std::string("abcd\xC3\xA9""fg"), pattern);

    std::locale::global(old);

    return result;
}

Update test_unicode() still works for me

$ file regex-test.cpp 
regex-test.cpp: UTF-8 Unicode c program text

$ g++ --version
Configured with: --prefix=/Applications/Xcode-8.2.1.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode-8.2.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

More Related Contents:

  1. Is gcc 4.8 or earlier buggy about regular expressions?
  2. UTF8 to/from wide char conversion in STL
  3. Unicode encoding for string literals in C++11
  4. How well is Unicode supported in C++11?
  5. What is the Windows equivalent for en_US.UTF-8 locale?
  6. What does std::match_results::size return?
  7. Difference between std::regex_match & std::regex_search?
  8. Read Unicode UTF-8 file into wstring
  9. regex replace with callback in c++11?
  10. 😃 (and other Unicode characters) in identifiers not allowed by g++
  11. Using regex lookbehinds in C++11
  12. What does “ provide, and where is it documented?
  13. No matches with c++11 regex [duplicate]
  14. Is this C++11 regex error me or the compiler?
  15. C++ & Boost: encode/decode UTF-8
  16. Should I use wchar_t when using UTF-8?
  17. Convert between string, u16string & u32string
  18. c++11 regex slower than python
  19. Inheriting and overriding functions of a std::string?
  20. Regex not working as expected with C++ regex_match
  21. C++ regex_match not working
  22. How to initialize std::array elegantly if T is not default constructible?
  23. Unicode Processing in C++
  24. Is it possible to determine if a type is a scoped enumeration type?
  25. Is there actually a reason why overloaded && and || don’t short circuit?
  26. Waiting for multiple futures?
  27. What is the class keyword before a function argument?
  28. When must you pass io_context to boost::asio::spawn? (C++)
  29. Why aren’t there compiler-generated swap() methods in C++0x?
  30. Why must an enumeration’s size be provided when it is forward declared?

Leave a Comment