What factors make PHP Unicode-incompatible?

by

When PHP was started several years ago, UTF-8 was not really supported. We are talking about a time when non-Unicode OS like Windows 98/Me was still current and when other big languages like Delphi were also non-Unicode. Not all languages were designed with Unicode in mind from day 1, and completely changing your language to Unicode without breaking a lot of stuff is hard. Delphi only became Unicode compatible a year or two ago for example, while other languages like Java or C# were designed in Unicode from Day 1.

So when PHP grew and became PHP 3, PHP 4 and now PHP 5, simply no one decided to add Unicode. Why? Presumably to keep compatible with existing scripts or because utf8_de/encode and mb_string already existed and work. I do not know for sure, but I strongly believe that it has something to do with organic growth. Features do not simply exist by default, they have to be written by someone, and that simply did not happen for PHP yet.

Edit: Ok, I read the question wrong. The question is: How are strings stored internally? If I type in “Währung” or “Écriture”, which Encoding is used to create the bytes used? In case of PHP, it is ASCII with a Codepage. That means: If I encode the string using ISO-8859-15 and you decode it with some chinese codepage, you will get weird results. The alternative is in languages like C# or Java where everything is stored as Unicode, which means: There is no codepage anymore, and theoretically you cannot mess up. I recommend Joel’s article about Unicode and Character Sets, but essentially it boils down to: How are strings stored internally, and the answer with PHP is “Not in Unicode”, which means that you have to be very careful and explicit when processing strings to make sure to always keep the string in the proper encoding during input, storage (database) and output, which is very errorprone.

More Related Contents:

  1. How to decode Unicode escape sequences like “\u00ed” to proper UTF-8 encoded characters?
  2. Matching Unicode letter characters in PCRE/PHP
  3. Reference: Why are my “special” Unicode characters encoded weird using json_encode?
  4. Unicode character in PHP string
  5. preg_match and UTF-8 in PHP
  6. Fixing broken UTF-8 encoding
  7. Setting the PHP default encoding to UTF-8
  8. How to get code point number for a given character in a utf-8 string?
  9. PHP: Convert unicode codepoint to UTF-8
  10. How to extract text from the PDF document? [closed]
  11. Any way to return PHP `json_encode` with encode UTF-8 and not Unicode? [duplicate]
  12. Storing and displaying unicode string (हिन्दी) using PHP and MySQL
  13. PHP decoding and encoding json with unicode characters
  14. How to get the character from unicode code point in PHP?
  15. Convert all types of smart quotes with PHP
  16. UTF-8 to Unicode Code Points
  17. Multibyte trim in PHP?
  18. Exotic names for methods, constants, variables and fields – Bug or Feature?
  19. Trim unicode whitespace in PHP 5.2
  20. How to display Unicode data with PHP [duplicate]
  21. How do I convert Word smart quotes and em dashes in a string?
  22. convert Persian/Arabic numbers to English numbers
  23. strlen() and UTF-8 encoding
  24. What is normalized UTF-8 all about?
  25. How to open file in PHP that has unicode characters in its name?
  26. How to reverse a Unicode string
  27. Split string into sentences using regex
  28. PHP function imagettftext() and unicode
  29. Remove or match a Unicode Zero Width Space PHP
  30. UTF8 Filenames in PHP and Different Unicode Encodings

Leave a Comment