Why does strsplit use positive lookahead and lookbehind assertion matches differently?

by

I am not sure whether this qualifies as a bug, because I believe this is expected behaviour based on the R documentation. From ?strsplit:

The algorithm applied to each input string is

repeat {
    if the string is empty
        break.
    if there is a match
        add the string to the left of the match to the output.
        remove the match and all to the left of it.
    else
        add the string to the output.
        break.
}

Note that this means that if there is a match at the beginning of
a (non-empty) string, the first element of the output is ‘””’, but
if there is a match at the end of the string, the output is the
same as with the match removed.

The problem is that lookahead (and lookbehind) assertions are zero-length. So for example in this case:

FF <- "(?=funky)"
testString <- "take me to funky town"

gregexpr(FF,testString,perl=TRUE)
# [[1]]
# [1] 12
# attr(,"match.length")
# [1] 0
# attr(,"useBytes")
# [1] TRUE

strsplit(testString,FF,perl=TRUE)
# [[1]]
# [1] "take me to " "f"           "unky town"

What happens is that the lonely lookahead (?=funky) matches at position 12. So the first split includes the string up to position 11 (left of the match), and it is removed from the string, together with the match, which -however- has zero length.

Now the remaining string is funky town, and the lookahead matches at position 1. However there’s nothing to remove, because there’s nothing at the left of the match, and the match itself has zero length. So the algorithm is stuck in an infinite loop. Apparently R resolves this by splitting a single character, which incidentally is the documented behaviour when strspliting with an empty regex (when argument split=""). After this the remaining string is unky town, which is returned as the last split since there’s no match.

Lookbehinds are no problem, because each match is split and removed from the remaining string, so the algorithm is never stuck.

Admittedly this behaviour looks weird at first glance. Behaving otherwise however would violate the assumption of zero length for lookaheads. Given that the strsplit algorithm is documented, I belive this does not meet the definition of a bug.

More Related Contents:

  1. Extract a string between patterns/delimiters in R
  2. Extracting numbers from vectors of strings
  3. Extracting a string between other two strings in R
  4. Extract info inside all parenthesis in R
  5. Extract a regular expression match
  6. Remove all special characters from a string in R?
  7. Regex group capture in R with multiple capture-groups
  8. Create new column in dataframe based on partial string matching other column
  9. Remove all text between two brackets
  10. R – gsub replacing backslashes
  11. Remove numbers from alphanumeric characters
  12. Escaped Periods In R Regular Expressions
  13. R dplyr: rename variables using string functions
  14. Select rows from data.frame ending with a specific character string in R
  15. Complete word matching using grepl in R
  16. R split on delimiter (split) keep the delimiter (split)
  17. Find the location of a character in string
  18. Splitting a string on the first space
  19. Is there an R function to escape a string for regex characters
  20. regex for preserving case pattern, capitalization
  21. Test for numeric elements in a character string
  22. Remove square brackets from a string vector
  23. Are there raw strings in R for regular expressions?
  24. extract number after specific string
  25. Regex return file name, remove path and file extension
  26. Replace first occurrence of “:” but not second in R
  27. Extract last 4-digit number from a series in R using stringr
  28. Extracting text after last period in string [duplicate]
  29. How to measure similarity between strings?
  30. How to prevent regmatches drop non matches?

Leave a Comment