json_encode() non utf-8 strings?

by

Is there a way I can get json_encode() to work and display these characters instead of having to use utf8_encode() on all of my strings and ending up with stuff like “\u0082”?

If you have an ANSI encoded string, using utf8_encode() is the wrong function to deal with this. You need to properly convert it from ANSI to UTF-8 first. That will certainly reduce the number of Unicode escape sequences like \u0082 from the json output, but technically these sequences are valid for json, you must not fear them.

Converting ANSI to UTF-8 with PHP

json_encode works with UTF-8 encoded strings only. If you need to create valid json successfully from an ANSI encoded string, you need to re-encode/convert it to UTF-8 first. Then json_encode will just work as documented.

To convert an encoding from ANSI (more correctly I assume you have a Windows-1252 encoded string, which is popular but wrongly referred to as ANSI) to UTF-8 you can make use of the mb_convert_encoding() function:

$str = mb_convert_encoding($str, "UTF-8", "Windows-1252");

Another function in PHP that can convert the encoding / charset of a string is called iconv based on libiconv. You can use it as well:

$str = iconv("CP1252", "UTF-8", $str);

Note on utf8_encode()

utf8_encode() does only work for Latin-1, not for ANSI. So you will destroy part of your characters inside that string when you run it through that function.

Related: What is ANSI format?

For a more fine-grained control of what json_encode() returns, see the list of predifined constants (PHP version dependent, incl. PHP 5.4, some constants remain undocumented and are available in the source code only so far).

Changing the encoding of an array/iteratively (PDO comment)

As you wrote in a comment that you have problems to apply the function onto an array, here is some code example. It’s always needed to first change the encoding before using json_encode. That’s just a standard array operation, for the simpler case of pdo::fetch() a foreach iteration:

while($row = $q->fetch(PDO::FETCH_ASSOC))
{
  foreach($row as &$value)
  {
    $value = mb_convert_encoding($value, "UTF-8", "Windows-1252");
  }
  unset($value); # safety: remove reference
  $items[] = array_map('utf8_encode', $row );
}

More Related Contents:

  1. PHP json encode – Malformed UTF-8 characters, possibly incorrectly encoded [duplicate]
  2. Reference: Why are my “special” Unicode characters encoded weird using json_encode?
  3. Detect encoding and make everything UTF-8
  4. How can I write a file in UTF-8 format?
  5. Why does the PHP json_encode function convert UTF-8 strings to hexadecimal entities?
  6. Convert utf8-characters to iso-88591 and back in PHP
  7. ‘Malformed UTF-8 characters, possibly incorrectly encoded’ in Laravel
  8. How to handle user input of invalid UTF-8 characters
  9. How can I detect a malformed UTF-8 string in PHP?
  10. PHP messing with HTML Charset Encoding
  11. PHP json_encode json_decode UTF-8
  12. utf-8 special characters not displaying [duplicate]
  13. Ensuring valid UTF-8 in PHP
  14. The ultimate emoji encoding scheme
  15. UTF8 Filenames in PHP and Different Unicode Encodings
  16. How to keep json_encode() from dropping strings with invalid characters
  17. How do I detect if have to apply UTF-8 decode or encode on a string?
  18. how to use json_encode for my output?
  19. I keep getting these kind of errors in terms of HttpUrlConnection and BackGroundTask in android
  20. file_get_contents() Breaks Up UTF-8 Characters
  21. PHP Array to JSON Array using json_encode(); [duplicate]
  22. How to create a JSON object
  23. PHP returning JSON to JQUERY AJAX CALL
  24. How to best configure PHP to handle a UTF-8 website [duplicate]
  25. How to sort an array of UTF-8 strings?
  26. How to deal with backslashes in json strings php
  27. Regex to detect invalid UTF-8 string
  28. Using json_encode on objects in PHP (regardless of scope)
  29. How to decode a JSON String with several objects in PHP?
  30. Access object property with disallowed character in property name

Leave a Comment