character-encoding - w3toppers.com

How do I call the PowerShell CLI robustly, with respect to character encoding, input and output streams, quoting and escaping?

PowerShell CLI fundamentals: PowerShell editions: The CLI of the legacy, bundled-with-Windows Windows PowerShell edition is powershell.exe, whereas that of the cross-platform, install-on-demand PowerShell (Core) 7+ edition is pwsh.exe (just pwsh on Unix-like platforms). Interactive use: By default, unless code to execute is specified (via -Command (-c) or -File (-f, see below), an interactive session is … Read more

What’s the difference between UTF-8 and UTF-8 with BOM?

The UTF-8 BOM is a sequence of bytes at the start of a text stream (0xEF, 0xBB, 0xBF) that allows the reader to more reliably guess a file as being encoded in UTF-8. Normally, the BOM is used to signal the endianness of an encoding, but since endianness is irrelevant to UTF-8, the BOM is … Read more

SQL Server – defining an XML type column with UTF-8 encoding

Is there a way to define a SQL Server column/field as having UTF-8 encoding? No, the only Unicode encoding in SQL Server is UTF-16 Little Endian, which is how the NCHAR, NVARCHAR, NTEXT (deprecated as of SQL Server 2005 so don’t use this in new development; besides, it sucks compared to NVARCHAR(MAX) anyway), and XML … Read more

Read/write .txt file with special characters

It’s the output console which doesn’t support those characters. Since you’re using Eclipse, you need to ensure that it’s configured to use UTF-8 for this. You can do this by Window > Preferences > General > Workspace > Text File Encoding > set to UTF-8. See also: Unicode – How to get the characters right? … Read more

How do I convert a byte array with null terminating character to a String in Java?

byte arr[] = … Charset charset = … // Find the position of the first zero byte int i; for (i = 0; i < arr.length && arr[i] != 0; i++) { } String str = new String(arr, 0, i, charSet); Notes: It is generally a good idea to use an explicit CharSet parameter so … Read more

Unable to change charset from ISO-8859-1 to UTF-8 in glassfish 3.1

The -Dfile.encoding is a Oracle JVM specific setting as to how to read Java source files. This doesn’t have any influence on the charset as specified in the Content-Type header of the HTTP response. You need to add the following to your web.xml in order to send the response of all JSPs as UTF-8 and … Read more

does (w)ifstream support different encodings

C++ supports character encodings by means of std::locale and the facet std::codecvt. The general idea is that a locale object describes the aspects of the system that might vary from culture to culture, (human) language to language. These aspects are broken down into facets, which are template arguments that define how localization-dependent objects (include I/O … Read more

Android: Unicode/Charset problems when sending an SMS (sendTextMessage)

Ok, this seems to have been solved by simply using sendMultipartTextMessage instead of sendTextMessage for the messages. Who would’ve thought… it kind of makes sense because unicode characters use more “space” than “normal” ones.

In Java, how to find if first character in a string is upper case without regex

Assuming s is non-empty: Character.isUpperCase(s.charAt(0)) or, as mentioned by divec, to make it work for characters with code points above U+FFFF: Character.isUpperCase(s.codePointAt(0));

How to determine encoding table of a text file

If you’re on Linux, try file -i filename.txt. $ file -i vol34.tex vol34.tex: text/x-tex; charset=us-ascii For reference, here is my environment: $ which file /usr/bin/file $ file –version file-5.09 magic file from /etc/magic:/usr/share/misc/magic Some file versions (e.g. file-5.04 on OS X/macOS) have slightly different command-line switches: $ file -I vol34.tex vol34.tex: text/x-tex; charset=us-ascii $ file … Read more