Beautifulsoup 4: Remove comment tag and its content

So the page that I’m scrapping contains these html codes. How do I remove the comment tag <!– –> along with its content with bs4?

<div class="foo">
cat dog sheep goat
<p>NewPP limit report
Preprocessor node count: 478/300000
Post‐expand include size: 4852/2097152 bytes
Template argument size: 870/2097152 bytes
Expensive parser function count: 2/100
ExtLoops count: 6/100



You can use extract() (solution is based on this answer):

PageElement.extract() removes a tag or string from the tree. It
returns the tag or string that was extracted.

from bs4 import BeautifulSoup, Comment

data = """<div class="foo">
cat dog sheep goat

soup = BeautifulSoup(data)

div = soup.find('div', class_='foo')
for element in div(text=lambda text: isinstance(text, Comment)):

print soup.prettify()

As a result you get your div without comments:

<div class="foo">
    cat dog sheep goat