Reference: Why are my “special” Unicode characters encoded weird using json_encode?

by

First of all: There’s nothing wrong here. This is how characters can be encoded in JSON. It is in the official standard. It is based on how string literals can be formed in Javascript ECMAScript (section 7.8.4 “String Literals”) and is described as such:

Any code point may be represented as a hexadecimal number. The meaning of such a number is determined by ISO/IEC 10646. If the code point is in the Basic Multilingual Plane (U+0000 through U+FFFF), then it may be represented as a six-character sequence: a reverse solidus, followed by the lowercase letter u, followed by four hexadecimal digits that encode the code point. […] So, for example, a string containing only a single reverse solidus character may be represented as “\u005C”.

In short: Any character can be encoded as \u...., where .... is the Unicode code point of the character (or the code point of half of a UTF-16 surrogate pair, for characters outside the BMP).

"馬"
"\u99ac"

These two string literals represent the exact same character, they’re absolutely equivalent. When these string literals are parsed by a compliant JSON parser, they will both result in the string “馬”. They don’t look the same, but they mean the same thing in the JSON data encoding format.

PHP’s json_encode preferably encodes non-ASCII characters using \u.... escape sequences. Technically it doesn’t have to, but it does. And the result is perfectly valid. If you prefer to have literal characters in your JSON instead of escape sequences, you can set the JSON_UNESCAPED_UNICODE flag in PHP 5.4 or higher:

php > echo json_encode(['foo' => '馬'], JSON_UNESCAPED_UNICODE);
{"foo":"馬"}

To emphasise: this is just a preference, it is not necessary in any way to transport “Unicode characters” in JSON.

More Related Contents:

  1. How to decode Unicode escape sequences like “\u00ed” to proper UTF-8 encoded characters?
  2. preg_match and UTF-8 in PHP
  3. Fixing broken UTF-8 encoding
  4. PHP: Convert unicode codepoint to UTF-8
  5. Any way to return PHP `json_encode` with encode UTF-8 and not Unicode? [duplicate]
  6. Storing and displaying unicode string (हिन्दी) using PHP and MySQL
  7. PHP decoding and encoding json with unicode characters
  8. PHP json encode – Malformed UTF-8 characters, possibly incorrectly encoded [duplicate]
  9. UTF-8 to Unicode Code Points
  10. can I get the unicode value of a character or vise versa with php?
  11. Trim unicode whitespace in PHP 5.2
  12. ‘Malformed UTF-8 characters, possibly incorrectly encoded’ in Laravel
  13. strlen() and UTF-8 encoding
  14. json_encode() non utf-8 strings?
  15. PHP json_encode json_decode UTF-8
  16. The ultimate emoji encoding scheme
  17. UTF8 Filenames in PHP and Different Unicode Encodings
  18. Trim unicode whitespace in PHP
  19. How to keep json_encode() from dropping strings with invalid characters
  20. I keep getting these kind of errors in terms of HttpUrlConnection and BackGroundTask in android
  21. How to use cURL to get jSON data and decode the data?
  22. JSON encode MySQL results
  23. How to parse JSON and access results
  24. How to avoid echoing character 65279 in php?
  25. Parsing JSON with PHP
  26. How to generate JSON data with PHP?
  27. How to reverse a Unicode string
  28. Using json_encode on objects in PHP (regardless of scope)
  29. Split string into sentences using regex
  30. Access object property with disallowed character in property name

Leave a Comment