Skip to content

Instantly share code, notes, and snippets.

View psd's full-sized avatar

Paul Downey psd

View GitHub Profile
@psd
psd / arcgis.py
Created June 11, 2024 06:21
Download GeoJSON from an ArcGIS API
#!/usr/bin/env python
import sys
import json
from esridump.dumper import EsriDumper
url = sys.argv[1]
dumper = EsriDumper(url, fields=None)
@psd
psd / .gitignore
Last active June 3, 2024 19:49
Create Local Land Charges Programme project markdown
.venv
organisation.csv
llc.html
@psd
psd / README.md
Created May 21, 2024 15:55
Finding old versions of a deleted file
(green-belt-collection) $ git blame --date=iso $(git log -2 --oneline -- collection/endpoint.csv | tail -1 | cut -d' ' -f1) collection/endpoint.csv
^fde75b2 (Paul Downey 2021-05-28 14:56:37 +0100  1) endpoint,endpoint-url,parameters,plugin,entry-date,start-date,end-date
05167dc5 (Paul Downey 2022-03-09 15:51:04 +0000  2) 21f6ac0160488dfdab4fae54bea1940ec2827e9c80d3b8df7ea73a45229c000f,http://maps.communities.gov.uk/geoserver/dclg_inspire/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=dclg_inspire%3AEngland_Green_Belt_2019-20_WGS84&outputFormat=GML3,,wfs,2021-10-27T06:06:33Z,,
05167dc5 (Paul Downey 2022-03-09 15:51:04 +0000  3) cbe952dd437d085d8c630d86632924693bac236c4fbad92cfdb623db3313e564,http://maps.communities.gov.uk/geoserver/dclg_inspire/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=dclg_inspire%3AEngland_Green_Belt_2018-19_WGS84&outputFormat=GML3,,wfs,2021-10-27T07:07:14Z,,
05167dc5 (Paul Downey 2022-03-09 15:51:04 +0000  4) 5cf30af559859b8e01088a0765f5864115ff3fe5504508168f1
@psd
psd / .gitignore
Last active September 28, 2023 08:57
.venv
@psd
psd / Untitled.ipynb
Last active June 30, 2023 08:08
Example analysis of broken endpoints on planning.data.gov.uk
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@psd
psd / README.md
Created June 13, 2023 08:37
listed-buildings-collection

files from running the listed-buildings-collection build locally

title
Conservation area datasets
erDiagram
    conservation-area {
        string reference
        string name
        string geometry
 url documentation-url
@psd
psd / simplify.py
Last active February 1, 2023 10:12
Reduce precision of a MultiPolygon
#!/usr/bin/env python3
import shapely.wkt
p="MultiPolygon (((-1.60994665622805222 55.00379498345115792, -1.60994600619739314 55.00379491903674278, -1.60994613329638736 55.0037946152858126, -1.60994665622805222 55.00379498345115792)),((-1.60994600619739314 55.00379491903674278, -1.60980778882421394 55.00412523870480186, -1.60972186100530901 55.00432707234649143, -1.60962254155930573 55.00455676609992395, -1.60953000060672702 55.00476385735949236, -1.60943341282917385 55.00498224763406085, -1.60935696567569031 55.00515319174433415, -1.60910274172032675 55.0051207020774342, -1.60860998788058174 55.00506105056158646, -1.60855767448542109 55.00517852794069285, -1.60848120855383003 55.00535097971769716, -1.60843013302326776 55.00547600239842438, -1.60833627909956944 55.00568233437196142, -1.60824905786570738 55.00588114622691904, -1.60818202334823357 55.00602798745143929, -1.60812690700974681 55.00616355492300613, -1.60809945369099605 55.00628789933396945, -1.60804720332470663 55.00639858943617355, -1.60796400348
@psd
psd / dns check.sh
Created November 24, 2022 11:02
Check www.planning.data.gov.uk DNS records
echo "==> Google \n$(dig www.planning.data.gov.uk @8.8.8.8 | grep "IN")\n\n"
echo "==> Google (alt) \n$(dig www.planning.data.gov.uk @8.8.4.4 | grep "IN")\n\n"
echo "==> Control D \n$(dig www.planning.data.gov.uk @76.76.2.0 | grep "IN")\n\n"
echo "==> Control D (alt) \n$(dig www.planning.data.gov.uk @76.76.10.0 | grep "IN")\n\n"
echo "==> Quad9 \n$(dig www.planning.data.gov.uk @9.9.9.9 | grep "IN")\n\n"
echo "==> Quad9 (alt) \n$(dig www.planning.data.gov.uk @149.112.112.112 | grep "IN")\n\n"
echo "==> OpenDNS \n$(dig www.planning.data.gov.uk @208.67.222.222 | grep "IN")\n\n"
echo "==> OpenDNS (alt) \n$(dig www.planning.data.gov.uk @208.67.220.220 | grep "IN")\n\n"
echo "==> Cloudflare \n$(dig www.planning.data.gov.uk @1.1.1.1 | grep "IN")\n\n"
echo "==> Cloudflare (alt) \n$(dig www.planning.data.gov.uk @1.0.0.1 | grep "IN")\n\n"
@psd
psd / notes.md
Last active November 4, 2022 13:44
QGIS and plannning.data.gov.uk

Some notes and thoughts from our QGIS workshop

  • Tony made a QGIS project for Doncaster's data
  • The QGIS project file is XML, which looks very easy for us to template / produce
  • We could make a QGIS project file for each LPA containing the data sources, alongside our processed data (filtered for the LPA)
  • We can organise these sources in the project in different ways, including hierachy
  • Tony showed using our tile server as an API which allows seeing the features
  • We should consider documenting our support for people using this as an API
  • The tile server provides our properties against each shape
  • but QGIS splits features when they overlap tile boundaries