Skip to content

Instantly share code, notes, and snippets.

import os
import translate.tools.pocount
import logging
logger = logging.getLogger(__name__)
ch = logging.StreamHandler()
ch.setLevel(logging.WARNING)
logger.addHandler(ch)
translate.tools.pocount.logger.addHandler(ch)
@Bjwebb
Bjwebb / licenses.py
Last active September 25, 2017 10:12
import pkg_resources
# https://stackoverflow.com/questions/19086030/can-pip-or-setuptools-distribute-etc-list-the-license-used-by-each-install
def get_pkg_licenses(pkg):
try:
lines = pkg.get_metadata_lines('METADATA')
except (KeyError, IOError):
lines = pkg.get_metadata_lines('PKG-INFO')
license = None
license_classifiers = []
@last-updated-datetime @xml:lang @default-currency @humanitarian @hierarchy @linked-data-uri iati-identifier reporting-org/@ref reporting-org/@type reporting-org/@secondary-reporter reporting-org/narrative/0 reporting-org/narrative/0/@xml:lang title/narrative/0 title/narrative/0/@xml:lang description/0/@type description/0/narrative/0 description/0/narrative/0/@xml:lang participating-org/0/@ref participating-org/0/@type participating-org/0/@role participating-org/0/@activity-id participating-org/0/narrative/0 participating-org/0/narrative/0/@xml:lang other-identifier/0/@ref other-identifier/0/@type other-identifier/0/owner-org/@ref other-identifier/0/owner-org/narrative/0 other-identifier/0/owner-org/narrative/0/@xml:lang activity-status/@code activity-date/0/@type activity-date/0/@iso-date activity-date/0/narrative/0 activity-date/0/narrative/0/@xml:lang contact-info/0/@type contact-info/0/organisation/narrative/0 contact-info/0/organisation/narrative/0/@xml:lang contact-info/0/department/narrative/0 contact-
@Bjwebb
Bjwebb / 0_xquery.xq
Last active February 28, 2024 22:38
<codelist><metadata/><codelist-items>{
for $org in /iati-organisations/iati-organisation
let $code := if ($org/organisation-identifier)
then $org/organisation-identifier/text()
else $org/iati-identifier/text()
return if ($code = $org/reporting-org/@ref/data()) then
<codelist-item>
<code>{$code}</code>
{if ($org/name/narrative)
then $org/name else
declare namespace output = "http://www.w3.org/2010/xslt-xquery-serialization";
declare option output:method "csv";
declare option output:csv "header=yes, separator=comma";
<csv>{
for $org in /iati-organisations/iati-organisation
return <record>
<organisation-identifier>{
if ($org/organisation-identifier)
then $org/organisation-identifier/text()
else $org/iati-identifier/text()
import json_merge_patch
import flattentool
import traceback
import requests
import json
import copy
import os
release_schema = requests.get('http://standard.open-contracting.org/schema/1__1__0/release-schema.json').json()
import csv
import json
releases = []
csv.field_size_limit(500 * 1024 * 1024)
with open('uk_contracts_finder2_ocds.csv') as fp:
reader = csv.DictReader(fp)
for row in reader:
@Bjwebb
Bjwebb / notes.md
Last active September 18, 2016 15:08
PYCon UK 2016 lighning talk on OCDS

$9,500,000,000,000

I'm going to start with a large number, 9.5 trillion dollars, which as a pile of dollar notes would stretch to the moon and back.

This is the total global spending through govermentment contracting, which is 15% of global GDP.

If you want an accountable govement, it's important to have public data about all this government contracting. Sadly this is often lacking at the moment -

@Bjwebb
Bjwebb / mapping.json
Last active January 21, 2016 14:07
2.02 CSV2IATI mapping template
{
"organisation": {
"reporting-org": {
"text": "",
"ref": "",
"type": ""
},
"default-currency": "",
"lang": "",
"contact-info": {
@Bjwebb
Bjwebb / gist:1ca1eb0ab50bd6578487
Created April 14, 2015 09:59
Output of salt-call -l debug --local state.sls gitbug (when run first time) https://github.com/saltstack/salt/issues/22514
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] Using cached minion ID from /etc/salt/minion_id: winslow
[DEBUG ] Configuration file path: /etc/salt/minion
[DEBUG ] Reading configuration from /etc/salt/minion
[DEBUG ] The `dmidecode` binary is not available on the system. GPU grains will not be available.
[DEBUG ] LazyLoaded jinja.render
[DEBUG ] LazyLoaded yaml.render
[DEBUG ] LazyLoaded jinja.render
[DEBUG ] LazyLoaded yaml.render
[DEBUG ] LazyLoaded state.sls