Skip to content

Instantly share code, notes, and snippets.

View pnorman's full-sized avatar

Paul Norman pnorman

View GitHub Profile
@pnorman
pnorman / detect_osm_weirdness.py
Created March 12, 2012 16:54 — forked from iandees/detect_osm_weirdness.py
An OSM "weirdness" detector: Reads minutely/hourly diffs and looks for oddly-shaped ways or ways that don't quite make sense.
#!/bin/python
import sys
import urllib2
import xml.etree.cElementTree as ElementTree
import simplejson as json
from datetime import datetime
import StringIO
import gzip
import time
@pnorman
pnorman / gist:5214260
Last active December 15, 2015 06:09
current html email message
Hi there!
Someone (hopefully you) just created an account at notes.apis.dev.openstreetmap.org.
Before we do anything else, we need to confirm that this request came from you, so if it did then please click the link below to confirm your account:
http://notes.apis.dev.openstreetmap.org/user/pnorman/confirm?confirm_string=...
We would like to welcome you and provide you with some additional information to get you started.
You can watch an introductory video to OpenStreetMap. There are more videos here.
Get reading about OpenStreetMap on the wiki, catch up with the latest news via the OpenStreetMap blog or Twitter, or browse through OpenStreetMap founder Steve Coast's OpenGeoData blog for the potted history of the project, which has podcasts to listen to also!
You can ask any questions you may have about OpenStreetMap at our question and answer site.
You may also want to sign up to the OpenStreetMap wiki.
It is recommended that you create a user wiki page, which includes category tags noting where you are, such
@pnorman
pnorman / gist:5214304
Created March 21, 2013 16:15
current text email message
Hi there!
Someone (hopefully you) just created an account at %{site_url}."
Before we do anything else, we need to confirm that this request came from you, so if it did then please click the link below to confirm your account:
<%= @url %>
We would like to welcome you and provide you with some additional information to get you started
You can watch an introductory video to OpenStreetMap here
Having reviewed 4f2a02a6 which adds this functionality, I'd classify the functions as a wrapper around the OSM API, with an OAuth implementation for authentication. It isn't immediately useful, but does remove the need for handling the details of the HTTP and OAuth requests.
What this API is not for is anyone just wanting to display a map with some data on it. It is for writing something to edit OpenStreetMap from within Joomla.
A few cautions for anyone planning on using it
You need to test software you write using it against api06.dev.openstreetmap.org before using it on the live API first.
The OSM API is provided for editing the map, see the API usage policy for more information
Building an OSM editor is complicated. What might be easier is a single-purpose editor, akin to WheelMap
If you're planning on adding data to OSM in bulk, you need to follow the Import Guidelines. I have seen too many people think the most useful then they can do with some OSM API interface is dump a large amount of geodata into
@pnorman
pnorman / gist:5790158
Created June 16, 2013 00:20
ec2 benchmark setup script
sudo apt-get update && sudo apt-get -y dist-upgrade && sudo shutdown -r now
# wait for reboot
# as root
apt-get autoremove -y --purge
# basics and osm2pgsql deps
apt-get install -y subversion git build-essential libxml2-dev libgeos-dev libgeos++-dev libpq-dev libbz2-dev libproj-dev protobuf-c-compiler libprotobuf-c0-dev autoconf automake libtool make g++ sysstat iotop apache2.2 apache2.2-common apache2-mpm-worker liblua5.2-dev lua5.2
mkdir -m 000 /planet
@pnorman
pnorman / List of OSM API calls
Created June 23, 2013 04:58
API calls list
API endpoint Request type Answerable from current planet + diffs
/api/capabilities GET No
/api/0.6/map GET Yes
/api/0.6/permissions GET No
/api/0.6/changeset/create PUT No
/api/0.6/changeset/#id GET No
/api/0.6/changeset/#id PUT No
/api/0.6/changeset/#id/close PUT No
/api/0.6/changeset/#id/download GET No
/api/0.6/changeset/#id/expand_bbox POST No
@pnorman
pnorman / errol osm2pgsql db size
Created June 25, 2013 08:36
size of errol's osm2pgsql database
osm2pgsql=> \d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+--------------------+-------+----------+------------+-------------
public | geography_columns | view | postgres | 0 bytes |
public | geometry_columns | view | postgres | 0 bytes |
public | planet_osm_line | table | apmon | 63 GB |
public | planet_osm_nodes | table | apmon | 8192 bytes |
public | planet_osm_point | table | apmon | 13 GB |
public | planet_osm_polygon | table | apmon | 56 GB |
@pnorman
pnorman / some apidb tests
Created June 26, 2013 21:02
apidb tests on join vs cgimap cache speed
-- tweak range and modulo value to get right number of nodes and changesets
select count(*) from (select distinct n.changeset_id from current_nodes n where n.id between
2329800000 and 2347300000 and n.id % 10 =0) as changesets;
select count(*) from current_nodes n where n.id between
2329800000 and 2347300000 and n.id % 10 =0;
-- JOIN
explain analyze select n.id, n.changeset_id, u.id, u.display_name from current_nodes
n join changesets c on n.changeset_id = c.id join users u on c.user_id =
@pnorman
pnorman / pqxxlimits.sql
Created June 30, 2013 02:17
outlines the cgimap nodes flow and some hstore limitations with libpqxx
CREATE TEMPORARY TABLE tmp_nodes (
id bigint primary key,
geom geometry, -- postgis geometry
-- actually other columns as well
tags hstore);
-- nodes has the same columns and indexes as tmp_nodes, bout about 2 billion rows
INSERT INTO tmp_nodes SELECT * FROM nodes WHERE ...; -- assorted WHERE clauses
-- more inserts with joins, complicated conditions, etc
@pnorman
pnorman / nodes_from_way_nodes.sql
Last active December 19, 2015 05:49
Two options for backfilling nodes from ways in cgimap.
-- Both queries produce the same results.
-- tmp_nodes and tmp_ways populated by geometries intersecting ST_SetSRID(ST_MakeBox2D(ST_Point(-123.4387,49.3308),ST_Point(-123.3034,49.4248)),4326)
-- ~5k nodes, ~400 ways, ~600 backfilled nodes
-- 150k set from ST_SetSRID(ST_MakeBox2D(ST_Point(-123.2748,49.1929),ST_Point(-122.9850,49.3058)),4326)
-- subselect variant
EXPLAIN ANALYZE INSERT INTO tmp_nodes
SELECT n.id,n.version,n.user_id,n.tstamp,n.changeset_id,n.tags,
ST_X(n.geom) * 10000000, ST_Y(n.geom) * 10000000