is PHP str_word_count() multibyte safe?

by

I’d say you guess right. And indeed there are space characters in UTF-8 which are not part of US-ASCII. To give you an example of such spaces:

And perhaps as well:

Anyway, the first one – the ‘NO-BREAK SPACE’ (U+00A0) – is a good example as it is also part of Latin-X charsets. And the PHP manual already provides a hint that str_word_count would be locale dependent.

If we want to put this to a test, we can set the locale to UTF-8, pass in an invalid string containing a \xA0 sequence and if this still counts as word-breaking character, that function is clearly not UTF-8 safe, hence not multibyte safe (as same non-defined as per the question):

<?php
/**
 * is PHP str_word_count() multibyte safe?
 * @link https://stackoverflow.com/q/8290537/367456
 */

echo 'New Locale: ', setlocale(LC_ALL, 'en_US.utf8'), "\n\n";

$test   = "aword\xA0bword aword";
$result = str_word_count($test, 2);

var_dump($result);

Output:

New Locale: en_US.utf8

array(3) {
  [0]=>
  string(5) "aword"
  [6]=>
  string(5) "bword"
  [12]=>
  string(5) "aword"
}

As this demo shows, that function totally fails on the locale promise it gives on the manual page (I do not wonder nor moan about this, most often if you read that a function is locale specific in PHP, run for your life and find one that is not) which I exploit here to demonstrate that it by no means does anything regarding the UTF-8 character encoding.

Instead for UTF-8 you should take a look into the PCRE extension:

PCRE has a good understanding of Unicode and UTF-8 in PHP in specific. It can also be quite fast if you craft the regular expression pattern carefully.

More Related Contents:

  1. How do I remove  from the beginning of a file?
  2. Reference: Why are my “special” Unicode characters encoded weird using json_encode?
  3. PHP DOMDocument loadHTML not encoding UTF-8 correctly
  4. SET NAMES utf8 in MySQL?
  5. How do I use filesystem functions in PHP, using UTF-8 strings?
  6. How to remove all non printable characters in a string?
  7. Detect encoding and make everything UTF-8
  8. UTF-8 BOM signature in PHP files
  9. Elegant way to search for UTF-8 files with BOM?
  10. Convert utf8-characters to iso-88591 and back in PHP
  11. PHP: Convert unicode codepoint to UTF-8
  12. Unable to retrieve UTF-8 accented characters from Access via PDO_ODBC
  13. Getting ’ instead of an apostrophe(‘) in PHP
  14. How to set UTF-8 encoding for a PHP file
  15. Regex to detect invalid UTF-8 string
  16. php header excel and utf-8
  17. json_encode() non utf-8 strings?
  18. UTF-8 encoding for subject in contact form email
  19. How to iterate UTF-8 string in PHP?
  20. URL Decoding in PHP
  21. How can I detect a malformed UTF-8 string in PHP?
  22. php substr() function with utf-8 leaves � marks at the end
  23. How to check if letter is upper or lower in PHP?
  24. utf8_(en|de)code removed from php7?
  25. PHP json_encode json_decode UTF-8
  26. fwrite() and UTF8
  27. UTF-8 not working in HTML forms
  28. Decoding numeric html entities via PHP
  29. Send emails with international accent and special characters
  30. Trim unicode whitespace in PHP

Leave a Comment