BeautifulSoup getting href [duplicate]

by

You can use find_all in the following way to find every a element that has an href attribute, and print each one:

from BeautifulSoup import BeautifulSoup

html=""'<a href="https://stackoverflow.com/questions/5815747/some_url">next</a>
<span class="class"><a href="another_url">later</a></span>'''

soup = BeautifulSoup(html)

for a in soup.find_all('a', href=True):
    print "Found the URL:", a['href']

The output would be:

Found the URL: some_url
Found the URL: another_url

Note that if you’re using an older version of BeautifulSoup (before version 4) the name of this method is findAll. In version 4, BeautifulSoup’s method names were changed to be PEP 8 compliant, so you should use find_all instead.

If you want all tags with an href, you can omit the name parameter:

href_tags = soup.find_all(href=True)

More Related Contents:

  1. BeautifulSoup: extract text from anchor tag
  2. How to find the comment tag with BeautifulSoup?
  3. Using Find_All function returns an unexpected result set
  4. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  5. Extracting an attribute value with beautifulsoup
  6. How to find elements by class
  7. How to remove \xa0 from string in Python?
  8. Scraping: SSL: CERTIFICATE_VERIFY_FAILED error for http://en.wikipedia.org
  9. Difference between .string and .text BeautifulSoup
  10. Python BeautifulSoup extract text between element
  11. How can I get href links from HTML using Python?
  12. Python/BeautifulSoup – how to remove all tags from an element?
  13. How to find tags with only certain attributes – BeautifulSoup
  14. Using BeautifulSoup to search HTML for string
  15. Extract content of with BeautifulSoup
  16. python beautifulsoup iframe document html extract
  17. How to change tag name with BeautifulSoup?
  18. How to find all comments with Beautiful Soup
  19. Can bs4 get the dynamic content of a webpage if requests can’t?
  20. Extract the ‘src’ attribute from an ‘img’ tag using Beautiful Soup
  21. Is there an InnerText equivalent in BeautifulSoup?
  22. How find specific data attribute from html tag in BeautifulSoup4?
  23. BeautifulSoup – extracting attribute values
  24. BeautifulSoup: Get the contents of a specific table
  25. Scrape the absolute URL instead of a relative path in python
  26. Get document DOCTYPE with BeautifulSoup
  27. Install Beautiful Soup using pip [duplicate]
  28. BeautifulSoup getText from between , not picking up subsequent paragraphs
  29. How to get inner text value of an HTML tag with BeautifulSoup bs4?
  30. How to scrape dynamic webpages by Python

Leave a Comment