Skip to content

Instantly share code, notes, and snippets.

View _get_musical_names.py
#!/usr/bin/python3
musical_notes = list('CDEFGAB')
#musical_notes = list('CDEFGAH')
fp = 'yob2020.txt'
with open(fp, 'r') as f:
lines = f.read().split('\n')
names = [x.split(',')[0].upper() for x in lines if len(x) > 0]
View get.py
import requests
df = pd.read_html(requests.get('https://www.slickcharts.com/sp500',headers={'User-agent': 'Mozilla/5.0'}).text)[0]
print(df)
df.to_csv('sp.csv',index=False)
@philshem
philshem / publibike_zuerich_elevation.csv
Created Feb 19, 2021
Elevation of Publibike stations around Zürich
View publibike_zuerich_elevation.csv
station_id latitude longitude elevation elevation_above_lake
pb_110 47.396401 8.4860017 399.0173645 -6.982635498
pb_149 47.392336 8.491061 399.4473877 -6.552612305
pb_286 47.3985553 8.493822 401.0709534 -4.929046631
pb_90 47.3917964 8.5056978 402.3087158 -3.69128418
pb_395 47.3669233 8.5420304 402.3295898 -3.670410156
pb_152 47.389823 8.49163 402.6643982 -3.335601807
pb_155 47.374711 8.543042 402.9417419 -3.058258057
pb_450 47.350578 8.560587 403.0110779 -2.988922119
pb_130 47.391366 8.488108 403.0958862 -2.90411377
@philshem
philshem / Collect Parler Metadata.ipynb
Created Jan 12, 2021 — forked from kylemcdonald/Collect Parler Metadata.ipynb
Collect video URLs and GPS data for Parler videos.
View Collect Parler Metadata.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@philshem
philshem / forbes400.csv
Last active Dec 7, 2020
scraper: Forbes' richest 400, 30 years
View forbes400.csv
We can't make this file beautiful and searchable because it's too large.
name,lastName,uri,imageUri,worthChange,age,source,industry,gender,timestamp,headquarters,state,realTimeWorth,date,realTimeRank,realTimePosition,squareImage,title,category,government,position,rank,worth,year,source_url,salary,managementAssets,pay
Abigail Johnson,Johnson,abigail-johnson,abigail-johnson,0.0,58.0,money management,Finance & Investments,F,2020-12-07 08:50:01.435,"Milton, MA",Massachusetts,14974.7,1993-01-01 00:00:00,39.0,39.0,//specials-images.forbesimg.com/imageserve/5f15ddd274642d00070464eb/416x416.jpg?background=000000&cropX1=62&cropX2=1355&cropY1=144&cropY2=1437,,,,,,,1990,http://www.forbes.com/ajax/list/data?year=1990&uri=forbes-400&type=person,,,
Aerin Lauder,Lauder,aerin-lauder,aerin-lauder,0.0,50.0,cosmetics,Fashion & Retail,F,2020-12-07 08:50:01.435,"New York, NY",New York,3215.528,1993-01-01 00:00:00,285.0,285.0,https://specials-images.forbesimg.com/imageserve/5f46b424808a289780083f05/416x416.jpg?background=000000&cropX1=895&cropX2=2304&cropY1=287&cropY2=1696,,,,,,,1990,http://www.forbes.
View deming.csv
x y y_ideal
10 10.9277728455939 10
10.25 10.5933490961181 10.25
10.5 14.0852031368451 10.5
10.75 12.4232922691896 10.75
11 14.7683460256395 11
11.25 9.93532391180117 11.25
11.5 12.4777201750146 11.5
11.75 11.3343548513887 11.75
12 10.2801940515687 12
View at.20200927.csv
We can make this file beautiful and searchable if this error is corrected: It looks like row 7 should actually have 11 columns, instead of 10. in line 6.
Datum,Tests,Fälle (Gesamt),Fälle (Aktiv),/ 100.000 Einwohner,Genesene,Verstorbene,Hospitalisiert,Intensivstation,state,url
27.09.2020 20:00,46.351 +476 (1%),758 +9 (1%),141 0 (0%),48,606 +9 (2%),11 +0 (0%),8 +1 (14%),1 0 (0%),burgenland,https://www.coronatracker.at/burgenland
26.09.2020 20:00,45.875 +592 (1%),749 +20 (3%),141 +6 (4%),48,597 +14 (2%),11 +0 (0%),7 +4 (133%),1 +1 (100%),burgenland,https://www.coronatracker.at/burgenland
25.09.2020 20:00,45.283 +908 (2%),729 +8 (1%),135 -5 (-4%),46,583 +13 (2%),11 +0 (0%),3 -4 (-57%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland
24.09.2020 20:00,44.375 +528 (1%),721 +4 (1%),140 -1 (-1%),48,570 +5 (1%),11 +0 (0%),7 +3 (75%),0 0 (0%),burgenland,https://www.coronatracker.at/burgenland
23.09.2020 20:00,43.847 +999 (2%),717 +33 (5%),141 +23 (19%),48,565 +10 (2%),11 +0 (0%),4 -3 (-43%),0 -1 (-100%),burgenland,https://www.coronatracker.at/burgenland
22.09.2020 20:00,42.848 +303 (1%),684 +14 (2%),118 +2 (2%),40,555 +12 (2%),11 +0 (0%),7 -2 (-22%),1 0 (0%),b
@philshem
philshem / 53827775.py
Created Sep 1, 2020
refactor stackoverflow answer to read unread emails and also fix for loop https://stackoverflow.com/a/53827775/2327328
View 53827775.py
import imaplib
import email
# adapted from: https://stackoverflow.com/a/53827775/2327328
def read_email_from_gmail():
mail = imaplib.IMAP4_SSL('imap.gmail.com')
mail.login('MYEMAIL','MYSECRET')
mail.select('inbox')
View 1298632074849587204.py
import pandas as pd
# https://burntsushi.net/stuff/worldcitiespop.csv
df = pd.read_csv('worldcitiespop.csv',low_memory=False)
df = df.query('Country == "us"')
print(len(df))
# prints 141989
@philshem
philshem / data.json
Last active Aug 24, 2020
sample data
View data.json
{
"1960":316.91,
"1961":317.64,
"1962":318.45,
"1963":318.99,
"1964":319.62,
"1965":320.04,
"1966":321.38,
"1967":322.16,
"1968":323.04,