If you need to use this on larger strings, multiple calls to Replace() can get inefficient pretty quickly. You may be better off rebuilding your string character-by-character: var map = new Dictionary<char, string>() { { ‘ä’, “ae” }, { ‘ö’, “oe” }, { ‘ü’, “ue” }, { ‘Ä’, “Ae” }, { ‘Ö’, “Oe” }, { … Read more
Personally I solved the same issue by adding after the MySQL connection code: mysql_set_charset(“utf8”); or for mysqli: mysqli_set_charset($conn, “utf8”); or the mysqli OOP equivalent: $conn->set_charset(“utf8”); And sometimes you’ll have to define the main php charset by adding this code: mb_internal_encoding(‘UTF-8’); On the client HTML side you have to add the following header data : <meta … Read more
Set up a collation using sqlite3_create_collation and then use it like this: SELECT * FROM users WHERE name LIKE “Andre%” COLLATE NOACCENTS
As usual, if you think of a problem which most certainly is not yours only, there’s already a solution on CPAN. ) In this case it’s called Text::Unidecode use warnings; use strict; use utf8; use Text::Unidecode; print unidecode(‘ä, ö, ü, é’); # will print ‘a, o, u, e’
You can change the collation at runtime in the sql query, …where title like ‘%torun%’ collate utf8_unicode_ci but beware that changing the collation on the fly at runtime forgoes the possibility of mysql using an index, so performance on large tables may be terrible. Or, you can copy the column to another column, such as … Read more
In modern browsers and node.js you can use unicode normalization to decompose those characters followed by a filtering regex. str.normalize(‘NFKD’).replace(/[^\w]/g, ”) If you wanted to allow characters such as whitespaces, dashes, etc. you should extend the regex to allow them. str.normalize(‘NFKD’).replace(/[^\w\s.-_\/]/g, ”) var str=”áàâäãéèëêíìïîóòöôõúùüûñçăşţ”; var asciiStr = str.normalize(‘NFKD’).replace(/[^\w]/g, ”); console.info(str, asciiStr); NOTES: This method does … Read more
JavaScript’s regex implementation is not Unicode-aware. It only knows the ‘word characters’ in standard low-byte ASCII, which does not include é or any other accented or non-English letters. Because é is not a word character to JS, é followed by a space can never be considered a word boundary. (It would match \b if used … Read more
Ultimately, you’re going to have to give up on the idea of “correct”, for this problem. Translating the string, no matter how you do it, destroys accuracy in the name of compatibility and readability. All three options are equally compatible, but #1 and #2 suffer in terms of readability. So just run with it and … Read more
Since no one has ever bothered to post the code to do this, here it is: // \p{Mn} or \p{Non_Spacing_Mark}: // a character intended to be combined with another // character without taking up extra space // (e.g. accents, umlauts, etc.). private readonly static Regex nonSpacingMarkRegex = new Regex(@”\p{Mn}”, RegexOptions.Compiled); public static string RemoveDiacritics(string text) … Read more
This should be useful which handles almost all the cases. function Unaccent($string) { return preg_replace(‘~&([a-z]{1,2})(?:acute|cedil|circ|grave|lig|orn|ring|slash|th|tilde|uml|caron);~i’, ‘$1’, htmlentities($string, ENT_COMPAT, ‘UTF-8’)); }