How to find all comments with Beautiful Soup

by

You can pass a function to find_all() to help it check whether the string is a Comment.

For example I have below html:

<body>
   <!-- Branding and main navigation -->
   <div class="Branding">The Science &amp; Safety Behind Your Favorite Products</div>
   <div class="l-branding">
      <p>Just a brand</p>
   </div>
   <!-- test comment here -->
   <div class="block_content">
      <a href="https://www.google.com">Google</a>
   </div>
</body>

Code:

from bs4 import BeautifulSoup as BS
from bs4 import Comment
....
soup = BS(html, 'html.parser')
comments = soup.find_all(string=lambda text: isinstance(text, Comment))
for c in comments:
    print(c)
    print("===========")
    c.extract()

the output would be:

Branding and main navigation 
============
test comment here
============

BTW, I think the reason why find_all('Comment') doesn’t work is (from BeautifulSoup document):

Pass in a value for name and you’ll tell Beautiful Soup to only consider tags with certain names. Text strings will be ignored, as will tags whose names that don’t match.

More Related Contents:

  1. How to find elements by class
  2. Beautiful Soup findAll doesn’t find them all
  3. Can I remove script tags with BeautifulSoup?
  4. BeautifulSoup findAll() given multiple classes?
  5. How can I get href links from HTML using Python?
  6. BeautifulSoup webscraping find_all( ): finding exact match
  7. How to load all entries in an infinite scroll at once to parse the HTML in python
  8. How to find children of nodes using BeautifulSoup
  9. BeautifulSoup innerhtml?
  10. Understand the Find() function in Beautiful Soup
  11. Convert a HTML Table to JSON
  12. python beautifulsoup iframe document html extract
  13. BeautifulSoup: extract text from anchor tag
  14. How to find tag with particular text with Beautiful Soup?
  15. How find specific data attribute from html tag in BeautifulSoup4?
  16. Exclude unwanted tag on Beautifulsoup Python
  17. How to install beautiful soup 4 with python 2.7 on windows
  18. How to find the comment tag with BeautifulSoup?
  19. BeautifulSoup return unexpected extra spaces
  20. BeautifulSoup: get css classes from html
  21. How to get inner text value of an HTML tag with BeautifulSoup bs4?
  22. How to scrape dynamic webpages by Python
  23. Using Find_All function returns an unexpected result set
  24. Beautiful Soup: ‘ResultSet’ object has no attribute ‘find_all’?
  25. python BeautifulSoup parsing table
  26. How to retrieve the values of dynamic html content using Python
  27. H14 error in heroku – “no web processes running”
  28. Remove a tag using BeautifulSoup but keep its contents
  29. How to extract a JSON object that was defined in a HTML page javascript block using Python?
  30. how to get text from within a tag, but ignore other child tags

Leave a Comment