Skip to content

Instantly share code, notes, and snippets.

@a4amaan
Created July 16, 2020 12:34
Show Gist options
  • Save a4amaan/44ae172d3451da5eb3a25d1c7d6b6ac5 to your computer and use it in GitHub Desktop.
Save a4amaan/44ae172d3451da5eb3a25d1c7d6b6ac5 to your computer and use it in GitHub Desktop.
Scrape Data from online CSV File
url = "https://raw.githubusercontent.com/owid/covid-19-data/master/public/data/testing/covid-testing-latest-data-source-details.csv"
with closing(requests.get(url, stream=True)) as r:
f = (line.decode('utf-8') for line in r.iter_lines())
reader = csv.reader(f, delimiter=',', quotechar='"')
line_count = 0
for row in reader:
if line_count == 0:
line_count += 1
else:
iso_code = row[0]
country = find(iso_code=iso_code)
date = row[2]
if country and date:
date = datetime.strptime(str(date), "%Y-%m-%d").date()
tests_total = row[8]
tests_per_thousand = row[9]
tests_total_change = row[10]
tests_per_thousand_change = row[11]
line_count += 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment