Skip to content

Instantly share code, notes, and snippets.

136 haveorg
114 nohaveorg
250 total
@Bjwebb
Bjwebb / largest_files
Last active December 26, 2015 20:49
Largest IATI XML files - this is not autogenerated atm find . -type f -size +1024k -exec ls -lh {} \; | awk '{ print $5" "$9 }' | grep -v .git | sort -nr
70M ./sida/998.xml
41M ./unops/iati_activity_MULTI.xml
25M ./minbuza_nl/iati-activities.xml
17M ./unicef/Activity_file.xml
16M ./globalgiving/activities.xml
16M ./dfid/DFID-Non-Specific-Country-NS.xml
15M ./eu/EU_QZA.xml
14M ./sida/289.xml
14M ./dfatd-maecd/dfatd-maecd_activities-activites
12M ./undp/global_projects.xml
@Bjwebb
Bjwebb / reporting-orgs
Last active December 26, 2015 21:39
for f in *; do echo $f; cat ../IATI-Registry-Refresher/ckan/$f | grep -oE '\{[^}]*publisher_iati_id[^}]*\}' | sed -e 's/.*"value": "\\"//' -e 's/\\".*//' | head -n 1; xmllint --xpath "//reporting-org/@ref" $f/* | sed 's/ \?ref="/\n/g' | sed 's/"$//' | tail -n +2 | sort | uniq -c; done > reporting-orgs
aa
21020
17 21020
aai
NL-KVK-27264198
15 NL-KVK-27264198
aauk
GB-CHC-274467
35 GB-CHC-274467
acaa
@Bjwebb
Bjwebb / -
Created November 1, 2013 10:43
{"download_errors":28, "publishers_nonopen_files":33}
aa-activity
aai-998
aai-af
aai-bd
aai-in
aai-mm
aai-mw
aai-ng
aai-np
aai-tz

//recipient-country

There should be at least one of::

@code

There should be at least one of::

@percentage
@Bjwebb
Bjwebb / reporting-org.csv
Created November 6, 2013 10:16
IFS=$'\n'; for f in *; do for line in `xmllint --xpath "//reporting-org/@ref" $f/* | sed 's/ \?ref="/\n/g' | sed 's/"$//' | tail -n +2 | sort | uniq -c`; do xmlid=`echo $line | awk '{ print $2 }'`; cat ../IATI-Registry-Refresher/ckan/$f | grep -oE '\{[^}]*publisher_iati_id[^}]*\}' | sed -e 's/.*"value": "\\"//' -e 's/\\".*//' | head -n 1 > tmp; …
acaa GB-CHC-1096908
apt GB-CHC-290836 GB-CHC-290836				
bracintl NL-CCI-20081098 NL-CCI-20081098
cafod GB-CC-285776 GB-CHC-285776
caid GB-CHC-110585 GB-CHC-1105851
caid GB-CHC-110585 GB-CHC-110858
cdc GB-2 GB-COH-03877777
cif 47134
cif 47135
cprojects GB-CC-326568 GB-CHC-326568
@Bjwebb
Bjwebb / -
Last active December 27, 2015 13:39
find . -type f -exec ls -l {} \; | awk '{ print $5" "$9 }' | grep -v .git | sort -n
114 ./sida/BG.xml
114 ./sida/BZ.xml
114 ./sida/CK.xml
114 ./sida/CU.xml
114 ./sida/CZ.xml
114 ./sida/GA.xml
114 ./sida/GY.xml
114 ./sida/HK.xml
114 ./sida/HU.xml
114 ./sida/IL.xml
@Bjwebb
Bjwebb / -
Created November 22, 2013 15:51
cat * | jq '.result[] | .license_id' | sort | uniq -c | gist
1 "attribution"
381 "cc-by"
13 "cc-by-sa"
5 "cc-zero"
79 "notspecified"
16 null
52 "odc-odbl"
82 "odc-pddl"
1563 "other-at"
5 "other-nc"
@Bjwebb
Bjwebb / -
Created November 25, 2013 17:21
1 "OSI Approved::Attribution Assurance Licenses"
381 "OKD Compliant::Creative Commons Attribution"
13 "OKD Compliant::Creative Commons Attribution-sa"
5 "OKD Compliant::Creative Commons CCZero"
79 "Other::License Not Specified"
16 null
52 "OKD Compliant::Open Data Commons Open Database License (ODbL)"
82 "OKD Compliant::Open Data Commons Public Domain Dedication and Licence (PDDL)"
1563 "OKD Compliant::Other (Attribution)"
5 "Non-OKD Compliant::Other (Non-Commercial)"