Skip to content

Instantly share code, notes, and snippets.

@clarmso

clarmso/get_links_from_page.py

Created September 17, 2018 01:48
Show Gist options
  • Save clarmso/969ea7d538b29f465759cdbb3822c9b6 to your computer and use it in GitHub Desktop.
Save clarmso/969ea7d538b29f465759cdbb3822c9b6 to your computer and use it in GitHub Desktop.
Download ZIP
get_links_from_page.py
Raw
get_links_from_page.py
#!/usr/bin/env python
from bs4 import BeautifulSoup
import requests
def get_links_from_page(url):
page_html = requests.get(url).content
soup = BeautifulSoup(page_html, 'html.parser')
return [elem['href']for elem in soup.find_all("a")]
links = get_links_from_page('https://www.google.ca')
print(links)
links = get_links_from_page('https://www.cbc.ca')
print(links)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment