Word boundary with words starting or ending with special characters gives unexpected results

by

See what a word boundary matches:

A word boundary can occur in one of three positions:

  • Before the first character in the string, if the first character is a word character.
  • After the last character in the string, if the last character is a word character.
  • Between two characters in the string, where one is a word character and the other is not a word character.

In your pattern }\b only matches if there is a word char after } (a letter, digit or _).

When you use (\W|$) you require a non-word or end of string explicitly.

A solution is adaptive word boundaries:

re.search(r'(?:(?!\w)|\b(?=\w)){}(?:(?<=\w)\b|(?<!\w))'.format(re.escape('Sortes\index[persons]{Sortes}')), 'test Sortes\index[persons]{Sortes} test')

Or equivalent:

re.search(r'(?!\B\w){}(?<!\w\B)'.format(re.escape('Sortes\index[persons]{Sortes}')), 'test Sortes\index[persons]{Sortes} test')

Here, adaptive dynamic word boundaries are used that mean the following:

  • (?:(?!\w)|\b(?=\w)) (equal to (?!\B\w)) – a left-hand boundary, making sure the current position is at the word boundary if the next char is a word char, or no context restriction is applied if the next char is not a word char (note that you will need to use (?:\B(?!\w)|\b(?=\w)) if you want to disallow a word char immediately on the left if the next char is not a word char)
  • (?:(?<=\w)\b|(?<!\w)) (equal to (?<!\w\B)) – a right-hand boundary, making sure the current position is at the word boundary if the previous char is a word char, or no context restriction is applied if the previous char is not a word char (note that you will need to use (?:(?<=\w)\b|\B(?<!\w)) if you want to disallow a word char immediately on the right if the preceding char is not a word char).

You might also consider using unambiguous word boundaries based on negative lookarounds in these cases:

re.search(r'(?<!\w){}(?!\w)'.format(re.escape('Sortes\index[persons]{Sortes}')), 'test Sortes\index[persons]{Sortes} test')

Here, (?<!\w) negative lookbehind will fail the match if there is a word char immediately to the left of the current location, and (?!\w) negative lookahead will fail the match if there is a word char immediately to the right of the current location.

Which to choose? Adaptive word boundaries are more lenient compared to unambiguous word boundaries as the latter presume there must be no word chars on both ends of a match, while the former allow matching leading and trailing non-word chars in any context.

Note: It is easy to customize these lookaround patterns further (say, to only fail the match if there are letters around the pattern, use [^\W\d_] instead of \w, or if you only allow matches around whitespaces, use whitespace boundaries (?<!\S) / (?!\S) lookaround boundaries).

More Related Contents:

  1. How to find overlapping matches with a regexp?
  2. Regular expression to return text between parenthesis
  3. Checking whole string with a regex
  4. Create Pandas DataFrame from txt file with specific pattern
  5. How can I find all matches to a regular expression in Python?
  6. Easiest way to replace a string using a dictionary of replacements?
  7. Recursive pattern in regex
  8. Find longest repetitive sequence in a string
  9. Python Regex instantly replace groups
  10. How to remove parentheses and all data within using Pandas/Python?
  11. Regular Expressions: Search in list
  12. Return string with first match for a regex, handling case where there is no match
  13. What’s the cleanest way to extract URLs from a string using Python?
  14. Regular expression: match start or whitespace
  15. Check if string matches pattern
  16. Python – Using regex to find multiple matches and print them out [duplicate]
  17. Regular expression parsing a binary file?
  18. Remove duplicate chars using regex?
  19. Match any unicode letter?
  20. Python replace string pattern with output of function
  21. How can I make multiple replacements in a string using a dictionary?
  22. Python regex split without empty string
  23. Creating a custom categorized corpus in NLTK and Python
  24. Warning raised by inserting 4-byte unicode to mysql
  25. How do I verify that a string only contains letters, numbers, underscores and dashes? [duplicate]
  26. Python regex findall
  27. Python regex: matching a parenthesis within parenthesis
  28. Regexp finding longest common prefix of two strings
  29. Splitting a math expression string into tokens in Python
  30. Download all pdf files from a website using Python

Leave a Comment