removing characters of a specific unicode range from a string

by

In the regex pattern add the negation operator ^. For filtering printable characters you could use the following expression [^\\x00-\\x7F] and you should get the desired result.

import java.io.UnsupportedEncodingException;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class UTF8 {
    public static void main(String[] args) {
        String utf8tweet = "";
        try {
            byte[] utf8Bytes = "#Hello twitter  How are you?".getBytes("UTF-8");

            utf8tweet = new String(utf8Bytes, "UTF-8");

        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
        Pattern unicodeOutliers = Pattern.compile("[^\\x00-\\x7F]",
                Pattern.UNICODE_CASE | Pattern.CANON_EQ
                        | Pattern.CASE_INSENSITIVE);
        Matcher unicodeOutlierMatcher = unicodeOutliers.matcher(utf8tweet);

        System.out.println("Before: " + utf8tweet);
        utf8tweet = unicodeOutlierMatcher.replaceAll(" ");
        System.out.println("After: " + utf8tweet);
    }
}

Results in the following output:

Before: #Hello twitter  How are you?
After: #Hello twitter   How are you?

EDIT

To explain further, you could also keep expressing the range with the \u form in the following way [^\\u0000-\\u007F], which will match all the characters which are not the first 128 UNICODE characters (the same as before). If you want to extend the range to support extra characters, you can do so using the UNICODE character list here.

For example if you want to include vowels with accent (used in Spanish) you should extend the range to \u00FF, so you have [^\\u0000-\\u00FF] or [^\\x00-\\xFF]:

Before: #Hello twitter  How are you? á é í ó ú
After: #Hello twitter   How are you? á é í ó ú

More Related Contents:

  1. What is the regex to extract all the emojis from a string?
  2. Java Variable Substitution
  3. Search for keywords and replace them with their abbreviations [closed]
  4. Regex to match end-of-sentence markers [closed]
  5. Java regex email
  6. Splitting on comma outside quotes
  7. How to write a UTF-8 file with Java?
  8. How to use regular expressions to parse HTML in Java?
  9. How to split a string between letters and digits (or between digits and letters)?
  10. How to split a string, but also keep the delimiters?
  11. Whitespace Matching Regex – Java
  12. How to match string within parentheses (nested) in Java?
  13. How can I make Notepad to save text in UTF-8 without the BOM?
  14. Java; String replace (using regular expressions)?
  15. Java regular expressions and dollar sign
  16. Cancelling a long running regex match?
  17. Regular expression to match a backslash followed by a quote
  18. Why String.replaceAll() in java requires 4 slashes “\\\\” in regex to actually replace “\”?
  19. Write a file in UTF-8 using FileWriter (Java)?
  20. How to filter string for unwanted characters using regex?
  21. replace \n and \r\n with in java
  22. Problems reading/writing UTF-8 data in MySQL from Java using JDBC connector 5.1
  23. Regex: what is InCombiningDiacriticalMarks?
  24. Regex that Will Match a Java Method Declaration
  25. RegEx in Java: how to deal with newline
  26. Pattern to extract text between parenthesis
  27. Regex search with pattern containing (?:.|\s)*? takes increasingly long time
  28. Java Encode file to Base64 string To match with other encoded string
  29. Extract ip addresses from Strings using regex
  30. Validate if input string is a number between 0-255 using regex

Leave a Comment