You shouldn’t use ls like that and a for loop is not appropriate either. Also, the destination directory should be outside the source directory. mkdir /path/to/destination find . -type f -exec iconv -f iso-8859-1 -t utf-8 “{}” -o /path/to/destination/”{}” \; No need for a loop. The -type f option includes files and excludes directories. Edit: … Read more
You can add it manually by first echoing the bytes into the file: echo -ne ‘\xEF\xBB\xBF’ > names.utf8.csv and then concatenating your required information at the end: iconv -f CP1252 -t UTF-8 names.csv >> names.utf8.csv Note the >> rather than >.
First, note that it is not possible to detect whether text belongs to a specific undesired encoding. You can only check whether a string is valid in a given encoding. You can make use of the UTF-8 validity check that is available in preg_match [PHP Manual] since PHP 4.3.5. It will return 0 (with no … Read more
Use parse, but don’t evaluate the results: x1 <- ‘pretty\\u003D\\u003Ebig’ x2 <- parse(text = paste0(“‘”, x1, “‘”)) x3 <- x2[[1]] x3 # [1] “pretty=>big” is.character(x3) # [1] TRUE length(x3) # [1] 1
None of the above answers worked for me. The only way to work around this problem was to remove all non graphical characters (http://stat.ethz.ch/R-manual/R-patched/library/base/html/regex.html). The code is this simple usableText=str_replace_all(tweets$text,”[^[:graph:]]”, ” “)
ASCII is a subset of UTF-8, so all ASCII files are already UTF-8 encoded. The bytes in the ASCII file and the bytes that would result from “encoding it to UTF-8” would be exactly the same bytes. There’s no difference between them, so there’s no need to do anything. It looks like your problem is … Read more
What about the WordPress implementation? function remove_accents($string) { if ( !preg_match(‘/[\x80-\xff]/’, $string) ) return $string; $chars = array( // Decompositions for Latin-1 Supplement chr(195).chr(128) => ‘A’, chr(195).chr(129) => ‘A’, chr(195).chr(130) => ‘A’, chr(195).chr(131) => ‘A’, chr(195).chr(132) => ‘A’, chr(195).chr(133) => ‘A’, chr(195).chr(135) => ‘C’, chr(195).chr(136) => ‘E’, chr(195).chr(137) => ‘E’, chr(195).chr(138) => ‘E’, chr(195).chr(139) => … Read more
Add BOM: UTF-8 file_put_contents($myFile, “\xEF\xBB\xBF”. $content);