UTF8 Filenames in PHP and Different Unicode Encodings

by

Thanks to the tips given in the two answers I was able to poke around and find some methods for normalizing the different unicode decompositions of a given character. In the situation I was faced with I was accessing files created by a OS X Carbon application. It is a fairly popular application and thus its file names seemed to adhere to a specific unicode decomposition.

In PHP 5.3 a new set of functions was introduced that allows you to normalize a unicode string to a particular decomposition. Apparently there are four decomposition standards which you can decompose you unicode string into. Python has had unicode normalization capabilties since version 2.3 via unicode.normalize. This article on python’s handling of unicode strings was helpful in understanding encoding / string handling a bit better.

Here is a quick example on normalizing a unicode filepath:

filePath = unicodedata.normalize('NFD', filePath)

I found that the NFD format worked for all my purposes, I wonder if this is this is the standard decomposition for unicode filenames.

More Related Contents:

  1. How to decode Unicode escape sequences like “\u00ed” to proper UTF-8 encoded characters?
  2. Reference: Why are my “special” Unicode characters encoded weird using json_encode?
  3. Detect encoding and make everything UTF-8
  4. preg_match and UTF-8 in PHP
  5. Fixing broken UTF-8 encoding
  6. Setting the PHP default encoding to UTF-8
  7. How can I write a file in UTF-8 format?
  8. Convert utf8-characters to iso-88591 and back in PHP
  9. PHP: Convert unicode codepoint to UTF-8
  10. Storing and displaying unicode string (हिन्दी) using PHP and MySQL
  11. PHP json encode – Malformed UTF-8 characters, possibly incorrectly encoded [duplicate]
  12. UTF-8 to Unicode Code Points
  13. can I get the unicode value of a character or vise versa with php?
  14. Trim unicode whitespace in PHP 5.2
  15. strlen() and UTF-8 encoding
  16. How to handle user input of invalid UTF-8 characters
  17. json_encode() non utf-8 strings?
  18. How can I detect a malformed UTF-8 string in PHP?
  19. PHP messing with HTML Charset Encoding
  20. utf-8 special characters not displaying [duplicate]
  21. Ensuring valid UTF-8 in PHP
  22. Trim unicode whitespace in PHP
  23. How do I detect if have to apply UTF-8 decode or encode on a string?
  24. Why doesn’t Python recognize my utf-8 encoded source file?
  25. How to store the data in unicode in hindi language
  26. Regular expressions for a range of unicode points PHP
  27. How to display Unicode data with PHP [duplicate]
  28. How to open file in PHP that has unicode characters in its name?
  29. UTF-8 safe equivalent of ord or charCodeAt() in PHP
  30. Turkish characters are not displayed correctly [duplicate]

Leave a Comment