Skip to content

Instantly share code, notes, and snippets.

@bradtraversy
Created July 29, 2018 12:02
Show Gist options
  • Star 27 You must be signed in to star a gist
  • Fork 10 You must be signed in to fork a gist
  • Save bradtraversy/f2014a236646ff62dccfc9fe5d469ed5 to your computer and use it in GitHub Desktop.
Save bradtraversy/f2014a236646ff62dccfc9fe5d469ed5 to your computer and use it in GitHub Desktop.
Simple scraping of a blog
import requests
from bs4 import BeautifulSoup
from csv import writer
response = requests.get('http://codedemos.com/sampleblog/')
soup = BeautifulSoup(response.text, 'html.parser')
posts = soup.find_all(class_='post-preview')
with open('posts.csv', 'w') as csv_file:
csv_writer = writer(csv_file)
headers = ['Title', 'Link', 'Date']
csv_writer.writerow(headers)
for post in posts:
title = post.find(class_='post-title').get_text().replace('\n', '')
link = post.find('a')['href']
date = post.select('.post-date')[0].get_text()
csv_writer.writerow([title, link, date])
@BekBrace
Copy link

BekBrace commented Mar 1, 2020

Thank you Brad for the ultimate awesomeness

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment