Extracting words from a string, removing punctuation and returning a list with separated words

by

This has nothing to do with splitting and punctuation; you just care about the letters (and numbers), and just want a regular expression:

import re
def getWords(text):
    return re.compile('\w+').findall(text)

Demo:

>>> re.compile('\w+').findall('Hello world, my name is...James the 2nd!')
['Hello', 'world', 'my', 'name', 'is', 'James', 'the', '2nd']

If you don’t care about numbers, replace \w with [A-Za-z] for just letters, or [A-Za-z'] to include contractions, etc. There are probably fancier ways to include alphabetic-non-numeric character classes (e.g. letters with accents) with other regex.

I almost answered this question here: Split Strings with Multiple Delimiters?

But your question is actually under-specified: Do you want 'this is: an example' to be split into:

  • ['this', 'is', 'an', 'example']
  • or ['this', 'is', 'an', '', 'example']?

I assumed it was the first case.

[this’, ‘is’, ‘an’, example’] is what i want. is there a method without importing regex? If we can just replace the non ascii_letters with ”, then splitting the string into words in a list, would that work? – James Smith 2 mins ago

The regexp is the most elegant, but yes, you could this as follows:

def getWords(text):
    """
        Returns a list of words, where a word is defined as a
        maximally connected substring of uppercase or lowercase
        alphabetic letters, as defined by "a".isalpha()

        >>> get_words('Hello world, my name is... Élise!')  # works in python3
        ['Hello', 'world', 'my', 'name', 'is', 'Élise']
    """
    return ''.join((c if c.isalnum() else ' ') for c in text).split()

or .isalpha()

Sidenote: You could also do the following, though it requires importing another standard library:

from itertools import *

# groupby is generally always overkill and makes for unreadable code
# ... but is fun

def getWords(text):
    return [
        ''.join(chars)
            for isWord,chars in 
            groupby(' My name, is test!', lambda c:c.isalnum()) 
            if isWord
    ]

If this is homework, they’re probably looking for an imperative thing like a two-state Finite State Machine where the state is “was the last character a letter” and if the state changes from letter -> non-letter then you output a word. Don’t do that; it’s not a good way to program (though sometimes the abstraction is useful).

More Related Contents:

  1. User input integer list [duplicate]
  2. Why is it string.join(list) instead of list.join(string)?
  3. Remove empty strings from a list of strings
  4. Converting a string representation of a list into an actual list object [duplicate]
  5. What exactly does the .join() method do?
  6. Join a list of items with different types as string in Python
  7. How slow is Python’s string concatenation vs. str.join?
  8. Loop “Forgets” to Remove Some Items [duplicate]
  9. Find and replace string values in list
  10. Case insensitive ‘in’
  11. Alphabet range in Python
  12. How do I split a string into a list of characters?
  13. Splitting on last delimiter in Python string?
  14. Python split string into multiple string [duplicate]
  15. Confused about __str__ on list in Python [duplicate]
  16. Joining pairs of elements of a list [duplicate]
  17. Finding a substring within a list in Python [duplicate]
  18. Perform a string operation for every element in a Python list
  19. Convert string to list. Python [string.split() acting weird]
  20. Why python’s list slicing doesn’t produce index out of bound error? [duplicate]
  21. How to create a list with the characters of a string? [duplicate]
  22. Convert a columns of string to list in pandas
  23. How to apply __str__ function when printing a list of objects in Python [duplicate]
  24. How to convert a string with comma-delimited items to a list in Python?
  25. How to return all list elements of a given length?
  26. Adding a string to a list using augmented assignment
  27. Column of lists, convert list to string as a new column
  28. Deleting consonants from a string in Python
  29. How can I convert string values from a dictionary, into int/float datatypes?
  30. How does str(list) work?

Leave a Comment