Skip to content

Instantly share code, notes, and snippets.

@jpmckinney
jpmckinney / fix_invalid_json.py
Last active May 10, 2021
Fixes errors in invalid JSON. Running time is (length of input) x (number of errors).
View fix_invalid_json.py
#!/usr/bin/env python
import json
import sys
def run(filename):
with open(filename) as f:
s = f.read()
# This code will re-read the file from the beginning after fixing each error, which is slow. A better solution
@jpmckinney
jpmckinney / flaticon_to_iso2.py
Last active Apr 26, 2021
Renames flags from Flaticon to use ISO 3166-1 alpha 2 codes.
View flaticon_to_iso2.py
import os
import sys
from hdx.location.country import Country
# https://www.flaticon.com/packs/rectangular-country-simple-flags?word=flags
FLAG_PATH = 'path/to/flaticon/folder'
excluded = {
# Non-ISO states with limited recognition
View convert_recommonmark_to_myst.py
import json
import os
import re
import sys
from textwrap import dedent
tables_replacements = {
'```eval_rst': '',
'```': '',
}
@jpmckinney
jpmckinney / pelican_check_export.py
Last active Apr 26, 2021
Exports the descriptions of data quality checks from Pelican.
View pelican_check_export.py
# Generates:
# https://docs.google.com/document/d/1l5SL1hUR9n8IDYVLSYsU85hdB3L_u2e3iD-txQPc0zU/edit
# https://docs.google.com/spreadsheets/d/1WPPOSf7xM9LR3VlTGx7wyzc8q_5xQ3HyMoB8eaEWHDM/edit#gid=0
# Gist: https://gist.github.com/jpmckinney/f3bdbb62620a9974ba1ff254392f6b6d
# GitHub: https://github.com/open-contracting/pelican/issues/63
# CRM: https://crm.open-contracting.org/issues/5908
import csv
import json
@jpmckinney
jpmckinney / README.md
Last active Nov 30, 2019
Compare differences after making changes to Sphinx
View README.md

Written as part of https://github.com/open-contracting/standard_profile_template/issues/33

curl -O https://gist.githubusercontent.com/jpmckinney/ba38d531e245c175e254ea3bc4218001/raw/bbb015f613ba936b3b42f25c576e833d196e5181/diff-reduce.py
python diff-reduce.py
diff -rq build t | grep Only | sort
diff -ru -x '*.js' -x '.buildinfo' -x '_sources' build t | less
View framework-agreement-tmp-paraguay.json
{
"tender": {
"title": "Adquisición de Útiles de Oficina",
"id": "355477-adquisicion-utiles-oficina",
"procuringEntity": {
"id": "50",
"name": "Dirección Nacional de Contrataciones Públicas (DNCP)"
},
"procurementMethod": "open",
"procurementMethodDetails": "Licitación Pública Nacional",
@jpmckinney
jpmckinney / README.md
Created Jul 20, 2019
Test the performance of approaches to flattening JSON to CSV.
View README.md

simple-one-table.py

Building a dict by reference is faster than recursively building a dict (slowest) or list of tuples.

dict:

return {key: obj}
…
d = {}
View translate_markdown.py
import gettext
import os
import re
import sys
from collections import OrderedDict
from docutils import frontend
from docutils import nodes
from docutils.parsers import rst
from docutils.utils import new_document
@jpmckinney
jpmckinney / character-class.rb
Created Feb 14, 2017
Character class to match encoding errors
View character-class.rb
/[¡¢£¤¥¦§¨©ª¬­®¯°±²³´µ¶·¸¹º¼½¾¿ÂÃÅËŒœŠšŸŽžƒˆ˜‘‚„†‡•…‰‹›€™]/
@jpmckinney
jpmckinney / digital-government-networks.md
Last active Jan 31, 2017
Online and informal intergovernmental networks
View digital-government-networks.md

Slack communities