Skip to content

Instantly share code, notes, and snippets.

@hrbrmstr
Last active January 3, 2016 02:09
Show Gist options
  • Save hrbrmstr/8394171 to your computer and use it in GitHub Desktop.
Save hrbrmstr/8394171 to your computer and use it in GitHub Desktop.
Get the amsmeteors.org observations <table> into CSV format (Python version)
import urllib2
from bs4 import BeautifulSoup
soup = BeautifulSoup(urllib2.urlopen("http://www.amsmeteors.org/observations/").read())
rows = soup.findChildren('table')[0].findChildren
for row in t[0].findAll('tr')[1:]:
col = row.findAll('td')
ts = col[0].string.encode('ascii',errors='ignore').replace("\r\n\ \t","").strip()
lat, c, lon = col[1].string.encode('ascii',errors='ignore').split()
facing = col[2].string.encode('ascii',errors='ignore').strip("\r\n\t\ ")
method = col[3].string.encode('ascii',errors='ignore').strip("\r\n\t\ ")
alt = col[4].string.encode('ascii',errors='ignore').strip("\r\n\t\ ").replace(",","").replace("m","").replace("-","0")
record = (ts, l1, l2, facing, method, alt)
# replace print(...) with a file write
print(",".join(record))
2014-01-03 10:34:00,32.6401,-116.3207,East,Counting,818
2014-01-03 07:00:00,32.6401,-116.3207,East,Counting,1210
2014-01-01 16:35:00,32.6401,-116.3207,South,Counting,36
2013-12-28 08:15:00,32.6401,-116.3207,North,Counting,0
2013-12-28 21:20:00,32.6401,-116.3207,West,Counting,4
2013-12-29 06:05:00,32.6401,-116.3207,East,Counting,8
2013-12-28 20:25:00,32.6401,-116.3207,North,Counting,166
2013-08-12 09:25:00,32.6401,-116.3207,East,Counting,818
2013-08-11 09:45:00,32.6401,-116.3207,East,Counting,818
2013-08-09 09:57:00,32.6401,-116.3207,East,Counting,818
2013-01-03 11:22:00,32.6401,-116.3207,East,Counting,818
2013-12-14 08:10:00,32.6401,-116.3207,South,Counting,871
2013-12-14 10:45:00,32.6401,-116.3207,East,Counting,818
2013-12-14 05:02:00,32.6401,-116.3207,South,Counting,1558
2013-12-13 10:40:00,32.6401,-116.3207,East,Counting,818
2013-12-13 04:30:00,32.6401,-116.3207,South,Counting,280
2013-12-05 12:26:00,32.6401,-116.3207,East,Counting,831
2013-12-03 18:00:00,32.6401,-116.3207,North,Counting,5
2013-12-03 17:45:00,32.6401,-116.3207,East,Counting,2
2013-12-01 05:35:00,32.6401,-116.3207,East,Counting,1020
2013-11-29 18:00:00,32.6401,-116.3207,North,Counting,830
2013-11-21 01:34:00,32.6401,-116.3207,North,Counting,102
2013-11-15 05:30:00,32.6401,-116.3207,Unsure,Plotting,268
2013-11-08 10:20:00,32.6401,-116.3207,North,Counting,0
2013-11-08 10:20:00,32.6401,-116.3207,North,Counting,0
2013-11-08 10:20:00,32.6401,-116.3207,North,Counting,0
2013-07-29 00:00:00,32.6401,-116.3207,East,Counting,497
2013-11-06 08:01:00,32.6401,-116.3207,East,Counting,368
2013-11-02 10:12:00,32.6401,-116.3207,Unsure,Plotting,28
2013-11-02 22:30:00,32.6401,-116.3207,North,Counting,202
2013-10-16 22:47:00,32.6401,-116.3207,North,Counting,1279
2013-10-16 22:47:00,32.6401,-116.3207,North,Counting,1279
2013-10-17 09:00:00,32.6401,-116.3207,West,Counting,142
2013-10-12 04:00:00,32.6401,-116.3207,South,Counting,284
2013-10-12 00:15:00,32.6401,-116.3207,South,Counting,168
2013-10-02 08:00:00,32.6401,-116.3207,South,Counting,17
2013-10-01 12:00:00,32.6401,-116.3207,East,Counting,253
2013-09-28 06:30:00,32.6401,-116.3207,West,Counting,253
2013-09-14 12:10:00,32.6401,-116.3207,North,Counting,253
2013-09-30 18:15:00,32.6401,-116.3207,South,Counting,356
2013-09-27 09:27:00,32.6401,-116.3207,North,Counting,30
2013-09-29 11:15:00,32.6401,-116.3207,East,Counting,157
2013-09-26 05:40:00,32.6401,-116.3207,South,Counting,305
2013-09-26 07:02:00,32.6401,-116.3207,North,Counting,213
2013-09-22 06:36:00,32.6401,-116.3207,North,Counting,301
2013-09-19 07:25:00,32.6401,-116.3207,East,Counting,132
2013-08-12 04:00:00,32.6401,-116.3207,North,Counting,284
2013-08-11 04:00:00,32.6401,-116.3207,North,Counting,284
2013-08-12 05:00:00,32.6401,-116.3207,North,Counting,1210
2013-08-04 03:30:00,32.6401,-116.3207,South,Counting,1210
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment