Finding all links in Beautiful Soup

Beautiful Soup

chevron_right

Cookbooks

chevron_right

Attribute Cookbook

schedule Jul 1, 2022

Last updated

local_offer Python●Beautiful Soup

Example

Just as an example consider the following HTML document:


        
        
            
                
                
                    my_html = """
   <html>
          <a href="www.abc.com">Site A</a>
          <a href="www.def.com">Site B</a>
          <a>Site C</a>
   </html>
"""

soup = BeautifulSoup(my_html)

Use the find_all(tag_name) like so:


        
        
            
                
                
                    for a in soup.find_all("a"):
    print(a)
                
            
            <a href="www.abc.com">Site A</a>
<a href="www.def.com">Site B</a>
<a>Site C</a>

If you want valid links that contain an url, specify href=True:


        
        
            
                
                
                    for a in soup.find_all("a", href=True):
       print(a)
                
            
            <a href="www.abc.com">Site A</a>
<a href="www.def.com">Site B</a>

Notice how Site C was omitted since it does not contain the href attribute.

To just extract the URLs:


        
        
            
                
                
                    for a in soup.find_all('a', href=True):
    print(a['href'])
                
            
            www.abc.com
www.def.com

Published by Isshin Inada

Edited by 0 others

Did you find this page useful?

thumb_up

thumb_down

Comment

Citation

Share on Twitter

Share on Facebook

Ask a question or leave a feedback...

thumb_up

thumb_down

chat_bubble_outline

settings

Enjoy our search

Hit / to insta-search docs and recipes!