Skip to content

Instantly share code, notes, and snippets.

Avatar

Tim Herzog tgherzog

View GitHub Profile
@tgherzog
tgherzog / build.py
Last active Jun 10, 2021
Ranking hierarchy code and output for WBG data API
View build.py
import wbgapi as wb
import pandas as pd
import json
sources = wb.source.Series()
df = pd.DataFrame({'name': wb.source.Series()})
df['ranking'] = df.index.astype('int64') * 10
df.loc['2', 'ranking'] = 1
df.loc['63', 'ranking'] = 2
df.loc['16', 'ranking'] = 3
@tgherzog
tgherzog / apitest1.py
Last active Apr 9, 2021
World Bank API: indicator endpoint consistency test
View apitest1.py
'''
Script that iterates the API databases to check if the indicator endpoints are internally consistent
Usage:
apitest1.py [--deep] [ID]
Options:
--deep: Deep test (compare indicator lists not just counts)
View lookup-test.py
# Run this in a Jupyter Notebook
import wbgapi as wb
import pandas as pd
from pyquery import PyQuery # JQuery for Python, sort of
pd.options.display.max_rows = 300
# scrape a list of UN member states
View knowledge-base.md
@tgherzog
tgherzog / terre-biodiv.py
Last active May 22, 2019
terre-biodiv data uploader
View terre-biodiv.py
"""
Upload files to a terre-biodiv s3 bucket
The first form uploads files from a local directory to an S3 bucket
The second form uses an S3 bucket as the source
Either BUCKET or SRCBUCKET can include a path prefix to control the
copy operation, e.g., wbg-terre-biodiv/data
View pyddh_example_dataset_add.py
# from github.com/tgherzog/pyddh - be sure to get the dkan-2018-api branch
import ddh
ddh.load('ddh1stg.prod.acquia-sites.com')
ds = ddh.dataset.ds_template()
# populate required fields
ds['title'] = 'Python test'
ds['body'] = 'Blah blah blah'
@tgherzog
tgherzog / 01-overview.md
Last active Nov 5, 2018
How to Deposit Energy Datasets in DDH
View 01-overview.md
View start.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>World Bank E-Learning Course</title>
<link rel="stylesheet" href="style/css/normalize.css"/>
<link rel="stylesheet" href="style/css/solo-localized.css"/>
<link rel="stylesheet" href="style/css/syntax.css"/>
@tgherzog
tgherzog / iso3.json
Last active Jul 9, 2018
Provides a mapping of ISO3 country codes to DDH taxonomy terms
View iso3.json
{"yem": "278", "mnp": "200", "lso": "158", "uga": "263", "tkm": "260", "alb": "36", "ita": "142", "tto": "257", "nld": "191", "moz": "186", "eap": "847", "mna": "845", "tcd": "76", "mne": "183", "mng": "182", "cuw": "33", "bfa": "68", "nga": "197", "zmb": "279", "geo": "114", "hrv": "89", "gtm": "123", "imn": "140", "aus": "47", "jam": "143", "aut": "48", "ven": "271", "irl": "139", "mwi": "167", "fin": "107", "rwa": "218", "lcr": "842", "bih": "61", "cpv": "70", "tjk": "250", "pse": "276", "lca": "221", "ssd": "34", "ssa": "840", "nor": "201", "mhl": "172", "pak": "203", "png": "206", "wsm": "224", "zwe": "280", "gum": "122", "gbr": "266", "guy": "127", "cri": "87", "cmr": "72", "mda": "180", "eas": "847", "mdg": "166", "hti": "128", "aze": "49", "qat": "214", "lao": "155", "are": "265", "arg": "44", "sen": "228", "btn": "59", "mdv": "169", "arm": "45", "grd": "120", "nac": "844", "bmu": "58", "est": "102", "mus": "175", "esp": "239", "lux": "163", "irq": "138", "bdi": "69", "smr": "225", "per": "208", "blr"
@tgherzog
tgherzog / coverage.py
Last active Aug 10, 2020
Some data quality checks
View coverage.py
#!/usr/bin/python -u
#wbgapi module available here: https://github.com/tgherzog/wbgapi
"""
coverage.py produces quick density coverage stats for an indicator
Usage:
coverage.py [--verbose --sources --prefixes --bssi --gaps] [--start YEAR] [--income INC] [--region RGN] [--since YEARS] [--archived-dbs IDS] INDICATOR...