BeautifulSoup – search by text inside a tag

by

The problem is that your <a> tag with the <i> tag inside, doesn’t have the string attribute you expect it to have. First let’s take a look at what text="" argument for find() does.

NOTE: The text argument is an old name, since BeautifulSoup 4.4.0 it’s called string.

From the docs:

Although string is for finding strings, you can combine it with
arguments that find tags: Beautiful Soup will find all tags whose
.string matches your value for string. This code finds the tags
whose .string is “Elsie”:

soup.find_all("a", string="Elsie")
# [<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>]

Now let’s take a look what Tag‘s string attribute is (from the docs again):

If a tag has only one child, and that child is a NavigableString, the
child is made available as .string:

title_tag.string
# u'The Dormouse's story'

(…)

If a tag contains more than one thing, then it’s not clear what
.string should refer to, so .string is defined to be None:

print(soup.html.string)
# None

This is exactly your case. Your <a> tag contains a text and <i> tag. Therefore, the find gets None when trying to search for a string and thus it can’t match.

How to solve this?

Maybe there is a better solution but I would probably go with something like this:

import re
from bs4 import BeautifulSoup as BS

soup = BS("""
<a href="https://stackoverflow.com/customer-menu/1/accounts/1/update">
    <i class="fa fa-edit"></i> Edit
</a>
""")

links = soup.find_all('a', href="https://stackoverflow.com/customer-menu/1/accounts/1/update")

for link in links:
    if link.find(text=re.compile("Edit")):
        thelink = link
        break

print(thelink)

I think there are not too many links pointing to /customer-menu/1/accounts/1/update so it should be fast enough.

More Related Contents:

  1. BeautifulSoup webscraping find_all( ): finding exact match
  2. Using BeautifulSoup to find a HTML tag that contains certain text
  3. BeautifulSoup returns empty list when searching by compound class names
  4. Extract the ‘src’ attribute from an ‘img’ tag using Beautiful Soup
  5. Python regular expression for HTML parsing
  6. Download all pdf files from a website using Python
  7. How to understand regular expression with python?
  8. Beautiful Soup: ‘ResultSet’ object has no attribute ‘find_all’?
  9. python BeautifulSoup parsing table
  10. Python regex matching Unicode properties
  11. How can I remove text within parentheses with a regex?
  12. Handling backreferences to capturing groups in re.sub replacement pattern
  13. Only extracting text from this element, not its children
  14. Python regular expressions – how to capture multiple groups from a wildcard expression?
  15. Replace first occurrence of string in Python
  16. How to handle response encoding from urllib.request.urlopen() , to avoid TypeError: can’t use a string pattern on a bytes-like object
  17. BeautifulSoup innerhtml?
  18. Python regex string matching?
  19. Re.match() returns always none
  20. How to use regular expression in lxml xpath?
  21. Matching partial ids in BeautifulSoup
  22. BeautifulSoup – modifying all links in a piece of HTML?
  23. Python BeautifulSoup: wildcard attribute/id search
  24. How does python and the regex module handle backslashes?
  25. How to tell if one regular expression matches a subset of another regular expression?
  26. how to get text from within a tag, but ignore other child tags
  27. Python regex findall
  28. Python regex: matching a parenthesis within parenthesis
  29. Clicking link using beautifulsoup in python
  30. Using Python and BeautifulSoup (saved webpage source codes into a local file)

Leave a Comment