How to use CSS selectors to retrieve specific links lying in some class using BeautifulSoup?

by

The page is not the most friendly in the use of classes and markup, but even so your CSS selector is too specific to be useful here.

If you want Upcoming Events, you want just the first <div class="events-horizontal">, then just grab the <div class="title"><a href="..."></div> tags, so the links on titles:

upcoming_events_div = soup.select_one('div.events-horizontal')
for link in upcoming_events_div.select('div.title a[href]'):
    print(link['href'])

Note that you should not use r.text; use r.content and leave decoding to Unicode to BeautifulSoup. See Encoding issue of a character in utf-8

More Related Contents:

  1. selenium.common.exceptions.InvalidSelectorException with “span:contains(‘string’)”
  2. BeautifulSoup: get css classes from html
  3. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  4. Scraping: SSL: CERTIFICATE_VERIFY_FAILED error for http://en.wikipedia.org
  5. Selenium Compound class names not permitted
  6. Difference between .string and .text BeautifulSoup
  7. Scraping Google Finance (BeautifulSoup)
  8. ImportError: No Module Named bs4 (BeautifulSoup)
  9. How to disable CSS in Python selenium using ChromeOptions
  10. Scrape a dynamic website
  11. Extract content of with BeautifulSoup
  12. How to scrape only visible webpage text with BeautifulSoup?
  13. Beautifulsoup : Difference between .find() and .select()
  14. Change the color of text within a pandas dataframe html table python using styles and css
  15. How to find all comments with Beautiful Soup
  16. How locate the pseudo-element ::before using Selenium Python
  17. How to find tag with particular text with Beautiful Soup?
  18. How can I access namespaced XML elements using BeautifulSoup?
  19. How find specific data attribute from html tag in BeautifulSoup4?
  20. CSS Problems with Flask Web App
  21. What causes `None` results from BeautifulSoup functions? How can I avoid “AttributeError: ‘NoneType’ object has no attribute…” with BeautifulSoup?
  22. Scrape the absolute URL instead of a relative path in python
  23. How to find the comment tag with BeautifulSoup?
  24. How can i grab CData out of BeautifulSoup
  25. Clicking link using beautifulsoup in python
  26. Beautiful Soup and Table Scraping – lxml vs html parser
  27. Using Python and BeautifulSoup (saved webpage source codes into a local file)
  28. How to override css in Django Admin?
  29. Download all pdf files from a website using Python
  30. How to handle IncompleteRead: in python

Leave a Comment