Multibyte trim in PHP?

by

The standard trim function trims a handful of space and space-like characters. These are defined as ASCII characters, which means certain specific bytes from 0 to 0100 0000.

Proper UTF-8 input will never contain multi-byte characters that is made up of bytes 0xxx xxxx. All the bytes in proper UTF-8 multibyte characters start with 1xxx xxxx.

This means that in a proper UTF-8 sequence, the bytes 0xxx xxxx can only refer to single-byte characters. PHP’s trim function will therefore never trim away “half a character” assuming you have a proper UTF-8 sequence. (Be very very careful about improper UTF-8 sequences.)

The \s on ASCII regular expressions will mostly match the same characters as trim.

The preg functions with the /u modifier only works on UTF-8 encoded regular expressions, and /\s/u match also the UTF8’s nbsp. This behaviour with non-breaking spaces is the only advantage to using it.

If you want to replace space characters in other, non ASCII-compatible encodings, neither method will work.

In other words, if you’re trying to trim usual spaces an ASCII-compatible string, just use trim. When using /\s/u be careful with the meaning of nbsp for your text.

Take care:

  $s1 = html_entity_decode(" Hello   "); // the NBSP
  $s2 = " 𩸽 exotic test ホ 𩸽 ";

  echo "\nCORRECT trim: [". trim($s1) ."], [".  trim($s2) ."]";
  echo "\nSAME: [". trim($s1) ."] == [". preg_replace('/^\s+|\s+$/','',$s1) ."]";
  echo "\nBUT: [". trim($s1) ."] != [". preg_replace('/^\s+|\s+$/u','',$s1) ."]";

  echo "\n!INCORRECT trim: [". trim($s2,'𩸽 ') ."]"; // DANGER! not UTF8 safe!
  echo "\nSAFE ONLY WITH preg: [". 
       preg_replace('/^[𩸽\s]+|[𩸽\s]+$/u', '', $s2) ."]";

More Related Contents:

  1. Matching Unicode letter characters in PCRE/PHP
  2. Regular expressions for a range of unicode points PHP
  3. Split string into sentences using regex
  4. PHP regular expressions: No ending delimiter ‘^’ found in
  5. Replace URLs in text with HTML links
  6. Fixing broken UTF-8 encoding
  7. How to remove accents and turn letters into “plain” ASCII characters? [duplicate]
  8. Regexp to add attribute in any xml tags
  9. PHP remove special character from string
  10. preg_replace: add number after backreference
  11. Retrieve all hashtags from a tweet in a PHP function
  12. How do I remove blank lines from text in PHP?
  13. How to use RegexIterator in PHP
  14. preg_match() Unknown modifier ‘[‘ help
  15. can I get the unicode value of a character or vise versa with php?
  16. Can someone explain the /e regex modifier? [duplicate]
  17. How do I replace double quotes with single quotes
  18. How to “Validate” Human Names in CakePHP?
  19. In PHP, how do I extract multiple e-mail addresses from a block of text and put them into an array?
  20. PHP regular expression to match lines starting with a special character
  21. Using regex to filter attributes in xpath with php
  22. Get last whole number in a string
  23. RegExp in preg_match function returning browser error
  24. Regex for password PHP [duplicate]
  25. regular expression and forward slash
  26. How to validate phone number using PHP? [duplicate]
  27. What does [\S\s]* mean in regex in PHP?
  28. UTF-8 in PHP regular expressions [duplicate]
  29. Get non-numeric characters then number on each line of a block of texf
  30. Validate date format in php

Leave a Comment