View rainfall.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
filenames = ['observatory-hill','parramatta','wollongong'] | |
# filenames = ['observatory-hill'] | |
# filename = 'lismore' | |
for filename in filenames: | |
df = pd.read_csv(f'{filename}.csv') | |
View getFiles.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
from zipfile import ZipFile | |
from io import BytesIO | |
import os | |
import time | |
from datetime import datetime | |
import schedule | |
url = "https://resultsxml.elections.qld.gov.au/Upload/publicResults.zip" |
View gist:f058390484f3b99bd4718faac2c35ce6
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
from zipfile import ZipFile | |
from io import BytesIO | |
import os | |
import time | |
from datetime import datetime | |
import schedule | |
timestamp = datetime.strftime(datetime.now(), '%Y%m%d%H%M%S') | |
url = "https://resultsxml.elections.qld.gov.au/Upload/publicResults.zip" |
View vic-lgas.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
import json | |
import scraperwiki | |
from datetime import datetime | |
url = 'https://wabi-australia-southeast-api.analysis.windows.net/public/reports/querydata?synchronous=true' | |
headers = { | |
"Accept": "application/json, text/plain, */*", | |
"ActivityId": "a1955107-c5d9-12b5-87a4-561001265ed6", |
View corona.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# This script will get the latest data from the Jons Hopkins GitHub repo https://github.com/CSSEGISandData | |
def getData(): | |
import requests | |
files = [ | |
"time_series_19-covid-Confirmed.csv", | |
"time_series_19-covid-Deaths.csv", | |
"time_series_19-covid-Recovered.csv" |
View demo.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
import scraperwiki | |
import math | |
# Function to save data into a SQLite database | |
def saveData(records): | |
scraperwiki.sqlite.save(unique_keys=["_id"], data=records) | |
# Get initial page of API results and create a json object |
View process.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#%% | |
import pandas as pd | |
import simplejson as json | |
from datetime import datetime | |
#%% | |
with open('1R3zG-DJRqN7MFLGqjhZs26D1SJcKI7pkd1j1XDwAiIM.json') as birdJson: | |
birdsInfo = json.load(birdJson)['sheets']['Sheet1'] |
View gist:348c2297152f20df369f0f219fc06e77
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
================================================================ | |
New ad from Liberal Party of Australia | |
Bill Shorten can't accurately answer simple questions about his new electric car policy. Yet he's pushing a plan that would drive up the cost of some cars by thousands of dollars. | |
https://www.facebook.com/LiberalPartyAustralia/ | |
================================================================ | |
New ad from ABC Friends | |
HOW WILL IT ALL END? Cartoon offering and a question from a great supporter of the ABC - Joan Rosser. PLEASE DONATE TO OUR ELECTION FUND SO WE CAN MAKE THE ABC CENTRAL TO THE ELECTION CAMPAIGN: https://www.mycause.com.au/page/198208/donate-to-make-the-abc-central-to-the-election | |
https://www.facebook.com/FriendsOfTheAbc/ | |
================================================================ | |
New ad from Australian Family Coalition |
View urls.csv
We can make this file beautiful and searchable if this error is corrected: It looks like row 5 should actually have 2 columns, instead of 1. in line 4.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
01-Premier-October-2018-December-2018.pdf,https://www.dpc.nsw.gov.au/assets/dpc-nsw-gov-au/publications/October-December-2018-1128/643cd59a5b/01-Premier-October-2018-December-2018.pdf | |
02-Treasurer-and-Minister-for-Industrial-Relations-October-2018-December-2018.pdf,https://www.dpc.nsw.gov.au/assets/dpc-nsw-gov-au/publications/October-December-2018-1128/987868a2d9/02-Treasurer-and-Minister-for-Industrial-Relations-October-2018-December-2018.pdf | |
Attorney-General-October-2018-December-2018.pdf,https://www.dpc.nsw.gov.au/assets/dpc-nsw-gov-au/publications/October-December-2018-1128/d2e1f8720f/Attorney-General-October-2018-December-2018.pdf | |
Deputy-Premier-Minister-for-Regional-NSW-Minister-for-Skills-Minister-for-Small-Business-October-2018-December-2018.pdf,https://www.dpc.nsw.gov.au/assets/dpc-nsw-gov-au/publications/October-December-2018-1128/040b8d52fe/Deputy-Premier-Minister-for-Regional-NSW-Minister-for-Skills-Minister-for-Small-Business-October-2018-December-2018.pdf | |
Minister-for-Counter-Terrorism-Minister- |
View read_pdfs.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import PyPDF2 | |
from os import listdir | |
from fuzzywuzzy import fuzz | |
import simplejson as json | |
files = listdir("docs/") | |
with open('template.txt', 'r') as f: | |
template=f.read().replace("\n", "") |
NewerOlder