python-re: How do I match an alpha character

by

Your first two sentences contradict each other. “in \w but is not in \d” includes underscore. I’m assuming from your third sentence that you don’t want underscore.

Using a Venn diagram on the back of an envelope helps. Let’s look at what we DON’T want:

(1) characters that are not matched by \w (i.e. don’t want anything that’s not alpha, digits, or underscore) => \W
(2) digits => \d
(3) underscore => _

So what we don’t want is anything in the character class [\W\d_] and consequently what we do want is anything in the character class [^\W\d_]

Here’s a simple example (Python 2.6).

>>> import re
>>> rx = re.compile("[^\W\d_]+", re.UNICODE)
>>> rx.findall(u"abc_def,k9")
[u'abc', u'def', u'k']

Further exploration reveals a few quirks of this approach:

>>> import unicodedata as ucd
>>> allsorts =u"\u0473\u0660\u06c9\u24e8\u4e0a\u3020\u3021"
>>> for x in allsorts:
...     print repr(x), ucd.category(x), ucd.name(x)
...
u'\u0473' Ll CYRILLIC SMALL LETTER FITA
u'\u0660' Nd ARABIC-INDIC DIGIT ZERO
u'\u06c9' Lo ARABIC LETTER KIRGHIZ YU
u'\u24e8' So CIRCLED LATIN SMALL LETTER Y
u'\u4e0a' Lo CJK UNIFIED IDEOGRAPH-4E0A
u'\u3020' So POSTAL MARK FACE
u'\u3021' Nl HANGZHOU NUMERAL ONE
>>> rx.findall(allsorts)
[u'\u0473', u'\u06c9', u'\u4e0a', u'\u3021']

U+3021 (HANGZHOU NUMERAL ONE) is treated as numeric (hence it matches \w) but it appears that Python interprets “digit” to mean “decimal digit” (category Nd) so it doesn’t match \d

U+2438 (CIRCLED LATIN SMALL LETTER Y) doesn’t match \w

All CJK ideographs are classed as “letters” and thus match \w

Whether any of the above 3 points are a concern or not, that approach is the best you will get out of the re module as currently released. Syntax like \p{letter} is in the future.

More Related Contents:

  1. Python regex matching Unicode properties
  2. Regex matching between two strings?
  3. remove unicode emoji using re in python
  4. How do I specify a range of unicode characters
  5. matching unicode characters in python regular expressions
  6. Python unicode regular expression matching failing with some unicode characters -bug or mistake?
  7. Matching only a unicode letter in Python re
  8. Regex and unicode
  9. Remove non-ASCII characters from a string using python / django
  10. re.findall behaves weird
  11. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  12. Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence
  13. Do regular expressions from the re module support word boundaries (\b)?
  14. Python extract pattern matches
  15. Remove all special characters, punctuation and spaces from string
  16. How can I convert surrogate pairs to normal string in Python?
  17. Natural sorting
  18. How do I check if a string is unicode or ascii?
  19. Removing u in list
  20. Python, remove all non-alphabet chars from string
  21. Python Regex to find a string in double quotes within a string
  22. Passing a function to re.sub in Python
  23. How to combine multiple regex into single one in python?
  24. pandas applying regex to replace values
  25. Python DictWriter writing UTF-8 encoded CSV files
  26. Using a RegEx to match IP addresses
  27. Whitespace gone from PDF extraction, and strange word interpretation
  28. re.search() only matches the first occurrence
  29. How do I verify that a string only contains letters, numbers, underscores and dashes? [duplicate]
  30. Regexp finding longest common prefix of two strings

Leave a Comment