OSMPoster

osmposter.md

Introduction

L'idée était d'utiliser les tracess gpx fournies par les contributeurs OSM. Je ne sais plus trop pourquoi j'étais parti la dessus, sans doute que c'était moins chiant et plus "fuzzy" que de tenter de récupérer tous les chemins possibles (routes, autoroutes, secondaires, tertiaires, pistes cyclables, ...), i.e. la donnée déjà traitée et valorisée dans la base OSM. J'avais voulu sans doute partir sur quelquechose un peu plus "brut de fonderie".

Donc il y avait plusieurs étapes:

1. récupération des traces GPX des contributeurs depuis l'API OSM
2. "sanitization" des traces, cf paragraphe d'après
3. Insertion dans postgis
4. Concaténation et extraction sous forme de shapefile
5. Ouverture du shapefile dans mapbox studio (et styling de la couche pour faire joli)
6. Export en PNG

"sanitization des GPX"

Il y a un point d'entrée de l'API pour ca (cf le script bash plus loin). Mais les traces GPX que les contributeurs OSM peuvent envoyer ont divers niveaux d'anonymisation (perso j'envoie les miennes en public, mais d'autres plus regardant préfèrent avoir les points des chemins de leur trace dans un ordre complètement aléatoire ...).

Donc j'ai écrit un utilitaire qui permet de remettre les points dans l'ordre "le plus probable".

L'outil sépare aussi en plusieurs traces si jamais la distance entre deux points est improbable pour un déplacement physique (vitesse instantannée entre les deux points supérieure à une contante dans le code, je limitais à 30m/s, cf ici).

A noter que j'avais d'abord commencé une implémentation en ruby, mais vu les performances catastrophiques et le temps de calcul, je me suis arrété en cours de route et j'ai recommencé en C.

Chargement des GPX en base postGIS

Donc on itère sur l'API OSM, chaque appel renvoie un GPX, et ce jusqu'à ce qu'il n'y ait plus de données à récupérer. Chaque GPX est ensuite nettoyé avec l'outil précédent.

L'étape suivante est de convertir les GPX nettoyés en dump PostGIS via ogr2ogr, puis de charger les fichiers dans une base. Pour cela, un schéma sql doit au préalable être chargé cf fichier osmposter_schema.sql, puis tous les fichiers générés précédemment.

Enfin, on extrait sous forme de shapefile les données de PostGIS via pgsql2shp.

MapBox Studio / Tilemill

On ouvre dans MapBox Studio notre shapefile, et on donne le style recherché à nos objets géographiques. J'avais personnellement utilisé du bleu avec un halo.

Il semble que MapBox Studio en version "client lourd" que j'utilisais au début de ce projet n'existe plus. Il a été remplacé par une application en ligne nécessitant un compte mapbox.

Toutefois, une version indépendante (et probablement maintenu communautairement en dehors de Mapbox) existe ici: https://github.com/tilemill-project/tilemill. Toutefois, elle s'éxecute comme une application backend et doit être pilotée par un navigateur web. A noter aussi que son installation avec nodeJS peut être un peu rébarbative.

Conclusions

Je ne sais plus trop pourquoi la chaine de traitement de données est aussi compliquée (d'abord GPX, puis sql / postGIS, pour finir avec un shapefile), mais il y a probablement plus simple.

osmposter.sh

#!/bin/sh

set -x 
# Chambéry
CITY=Chambéry
NORTH=45.5903
SOUTH=45.5354
WEST=5.8621
EAST=5.9755

# Geneva
#CITY=Genève
#NORTH=46.2303
#SOUTH=46.1783
#WEST=6.0846
#EAST=6.19888


rm -f ${CITY}.*.gpx
rm -rf ${CITY}
mkdir ${CITY}

# Gets the traces from the OSM API
COUNT=0
BREAK=0
while [ ${BREAK} -eq 0 ]; do
  wget -O ${CITY}.${COUNT}.gpx "http://api.openstreetmap.org/api/0.6/trackpoints?bbox=${WEST},${SOUTH},${EAST},${NORTH}&page=${COUNT}"
  if [ `cat ${CITY}.${COUNT}.gpx | wc -l` -eq 2 ] ; then BREAK=1 ; fi
  COUNT=$(($COUNT + 1))
done


# Setting up the GPX sanitizer
cd ./gpx_sanitizer/c_version ;
make install ;
cd ../../ ;
# If still unable to find it, abort.
if [ ! -f sanitize ]; then
  echo "Unable to find the GPX sanitizer tool. Aborting." ;
  exit ;
fi

# generates GPX -> pgdump

rm huge.sql
# sanitize tracks
for i in ${CITY}.*.gpx ; do
  ./sanitize $i ;
  for j in `basename ${i} .gpx`_*.gpx ; do
    mv $j ${CITY}/ ;
  done
done
cd ${CITY}
for i in *.gpx ; do
  ogr2ogr -lco DROP_TABLE=OFF -lco CREATE_TABLE=off -f PGDump `basename $i .gpx`.sql $i ;
  cat  `basename $i .gpx`.sql >> ../huge.sql ;
  rm `basename $i .gpx`.sql ;
done
cd ../

# Loads data into PostGIS
dropdb osmposter
createdb osmposter
psql osmposter < osmposter_schema.sql
psql osmposter < huge.sql
rm huge.sql

# Generates a shapefile
rm -rf ${CITY}_out_shp
mkdir  ${CITY}_out_shp
cd ${CITY}_out_shp
pgsql2shp osmposter tracks
cd ../


# don't really care actually
# Was meant to get the admin boundary of the city
#
#cat > oapi-${CITY}.req << EOF
#<union>
#  <query type="relation">
#    <has-kv k="name" v="${CITY}"/>
#    <has-kv k="admin_level" v="8"/>
#  </query>
#  <recurse type="relation-node" into="nodes"/>
#  <recurse type="relation-way"/>
#  <recurse type="way-node"/>
#</union>
#<print/>
#EOF
#
#wget --post-file=oapi-${CITY}.req --header='Content-Type: text/xml'  'http://api.openstreetmap.fr/oapi/interpreter' -O ${CITY}_bound.osm
#rm -f oapi-${CITY}.req
#osm2pgsql -d osmposter ${CITY}_bound.osm
#rm ${CITY}_bound.osm
#

osmposter_schema.sql

--
-- PostgreSQL database dump
--

SET statement_timeout = 0;
SET lock_timeout = 0;
SET client_encoding = 'UTF8';
SET standard_conforming_strings = on;
SET check_function_bodies = false;
SET client_min_messages = warning;

--
-- Name: plpgsql; Type: EXTENSION; Schema: -; Owner: 
--

CREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;


--
-- Name: EXTENSION plpgsql; Type: COMMENT; Schema: -; Owner: 
--

COMMENT ON EXTENSION plpgsql IS 'PL/pgSQL procedural language';


--
-- Name: postgis; Type: EXTENSION; Schema: -; Owner: 
--

CREATE EXTENSION IF NOT EXISTS postgis WITH SCHEMA public;


--
-- Name: EXTENSION postgis; Type: COMMENT; Schema: -; Owner: 
--

COMMENT ON EXTENSION postgis IS 'PostGIS geometry, geography, and raster spatial types and functions';


SET search_path = public, pg_catalog;

SET default_tablespace = '';

SET default_with_oids = false;

--
-- Name: route_points; Type: TABLE; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE TABLE route_points (
    ogc_fid integer NOT NULL,
    wkb_geometry geometry(Point,4326),
    route_fid integer,
    route_point_id integer,
    ele double precision,
    "time" timestamp with time zone,
    magvar double precision,
    geoidheight double precision,
    name character varying,
    cmt character varying,
    "desc" character varying,
    src character varying,
    url character varying,
    urlname character varying,
    sym character varying,
    type character varying,
    fix character varying,
    sat integer,
    hdop double precision,
    vdop double precision,
    pdop double precision,
    ageofdgpsdata double precision,
    dgpsid integer
);


--
-- Name: route_points_ogc_fid_seq; Type: SEQUENCE; Schema: public; Owner: pmauduit
--

CREATE SEQUENCE route_points_ogc_fid_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;



--
-- Name: route_points_ogc_fid_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: pmauduit
--

ALTER SEQUENCE route_points_ogc_fid_seq OWNED BY route_points.ogc_fid;


--
-- Name: routes; Type: TABLE; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE TABLE routes (
    ogc_fid integer NOT NULL,
    wkb_geometry geometry(LineString,4326),
    name character varying,
    cmt character varying,
    "desc" character varying,
    src character varying,
    link1_href character varying,
    link1_text character varying,
    link1_type character varying,
    link2_href character varying,
    link2_text character varying,
    link2_type character varying,
    number integer,
    type character varying
);


--
-- Name: routes_ogc_fid_seq; Type: SEQUENCE; Schema: public; Owner: pmauduit
--

CREATE SEQUENCE routes_ogc_fid_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;



--
-- Name: routes_ogc_fid_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: pmauduit
--

ALTER SEQUENCE routes_ogc_fid_seq OWNED BY routes.ogc_fid;


--
-- Name: track_points; Type: TABLE; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE TABLE track_points (
    ogc_fid integer NOT NULL,
    wkb_geometry geometry(Point,4326),
    track_fid integer,
    track_seg_id integer,
    track_seg_point_id integer,
    ele double precision,
    "time" timestamp with time zone,
    course double precision,
    speed double precision,
    magvar double precision,
    geoidheight double precision,
    name character varying,
    cmt character varying,
    "desc" character varying,
    src character varying,
    url character varying,
    urlname character varying,
    sym character varying,
    type character varying,
    fix character varying,
    sat integer,
    hdop double precision,
    vdop double precision,
    pdop double precision,
    ageofdgpsdata double precision,
    dgpsid integer
);


--
-- Name: track_points_ogc_fid_seq; Type: SEQUENCE; Schema: public; Owner: pmauduit
--

CREATE SEQUENCE track_points_ogc_fid_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;



--
-- Name: track_points_ogc_fid_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: pmauduit
--

ALTER SEQUENCE track_points_ogc_fid_seq OWNED BY track_points.ogc_fid;


--
-- Name: tracks; Type: TABLE; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE TABLE tracks (
    ogc_fid integer NOT NULL,
    wkb_geometry geometry(MultiLineString,4326),
    name character varying,
    cmt character varying,
    "desc" character varying,
    src character varying,
    link1_href character varying,
    link1_text character varying,
    link1_type character varying,
    link2_href character varying,
    link2_text character varying,
    link2_type character varying,
    number integer,
    type character varying
);


--
-- Name: tracks_ogc_fid_seq; Type: SEQUENCE; Schema: public; Owner: pmauduit
--

CREATE SEQUENCE tracks_ogc_fid_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;



--
-- Name: tracks_ogc_fid_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: pmauduit
--

ALTER SEQUENCE tracks_ogc_fid_seq OWNED BY tracks.ogc_fid;


--
-- Name: waypoints; Type: TABLE; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE TABLE waypoints (
    ogc_fid integer NOT NULL,
    wkb_geometry geometry(Point,4326),
    ele double precision,
    "time" timestamp with time zone,
    magvar double precision,
    geoidheight double precision,
    name character varying,
    cmt character varying,
    "desc" character varying,
    src character varying,
    url character varying,
    urlname character varying,
    sym character varying,
    type character varying,
    fix character varying,
    sat integer,
    hdop double precision,
    vdop double precision,
    pdop double precision,
    ageofdgpsdata double precision,
    dgpsid integer
);



--
-- Name: waypoints_ogc_fid_seq; Type: SEQUENCE; Schema: public; Owner: pmauduit
--

CREATE SEQUENCE waypoints_ogc_fid_seq
    START WITH 1
    INCREMENT BY 1
    NO MINVALUE
    NO MAXVALUE
    CACHE 1;



--
-- Name: waypoints_ogc_fid_seq; Type: SEQUENCE OWNED BY; Schema: public; Owner: pmauduit
--

ALTER SEQUENCE waypoints_ogc_fid_seq OWNED BY waypoints.ogc_fid;


--
-- Name: ogc_fid; Type: DEFAULT; Schema: public; Owner: pmauduit
--

ALTER TABLE ONLY route_points ALTER COLUMN ogc_fid SET DEFAULT nextval('route_points_ogc_fid_seq'::regclass);


--
-- Name: ogc_fid; Type: DEFAULT; Schema: public; Owner: pmauduit
--

ALTER TABLE ONLY routes ALTER COLUMN ogc_fid SET DEFAULT nextval('routes_ogc_fid_seq'::regclass);


--
-- Name: ogc_fid; Type: DEFAULT; Schema: public; Owner: pmauduit
--

ALTER TABLE ONLY track_points ALTER COLUMN ogc_fid SET DEFAULT nextval('track_points_ogc_fid_seq'::regclass);


--
-- Name: ogc_fid; Type: DEFAULT; Schema: public; Owner: pmauduit
--

ALTER TABLE ONLY tracks ALTER COLUMN ogc_fid SET DEFAULT nextval('tracks_ogc_fid_seq'::regclass);


--
-- Name: ogc_fid; Type: DEFAULT; Schema: public; Owner: pmauduit
--

ALTER TABLE ONLY waypoints ALTER COLUMN ogc_fid SET DEFAULT nextval('waypoints_ogc_fid_seq'::regclass);


--
-- Name: route_points_pk; Type: CONSTRAINT; Schema: public; Owner: pmauduit; Tablespace: 
--

ALTER TABLE ONLY route_points
    ADD CONSTRAINT route_points_pk PRIMARY KEY (ogc_fid);


--
-- Name: routes_pk; Type: CONSTRAINT; Schema: public; Owner: pmauduit; Tablespace: 
--

ALTER TABLE ONLY routes
    ADD CONSTRAINT routes_pk PRIMARY KEY (ogc_fid);


--
-- Name: track_points_pk; Type: CONSTRAINT; Schema: public; Owner: pmauduit; Tablespace: 
--

ALTER TABLE ONLY track_points
    ADD CONSTRAINT track_points_pk PRIMARY KEY (ogc_fid);


--
-- Name: tracks_pk; Type: CONSTRAINT; Schema: public; Owner: pmauduit; Tablespace: 
--

ALTER TABLE ONLY tracks
    ADD CONSTRAINT tracks_pk PRIMARY KEY (ogc_fid);


--
-- Name: waypoints_pk; Type: CONSTRAINT; Schema: public; Owner: pmauduit; Tablespace: 
--

ALTER TABLE ONLY waypoints
    ADD CONSTRAINT waypoints_pk PRIMARY KEY (ogc_fid);

--
-- Name: route_points_geom_idx; Type: INDEX; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE INDEX route_points_geom_idx ON route_points USING gist (wkb_geometry);


--
-- Name: routes_geom_idx; Type: INDEX; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE INDEX routes_geom_idx ON routes USING gist (wkb_geometry);


--
-- Name: track_points_geom_idx; Type: INDEX; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE INDEX track_points_geom_idx ON track_points USING gist (wkb_geometry);


--
-- Name: tracks_geom_idx; Type: INDEX; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE INDEX tracks_geom_idx ON tracks USING gist (wkb_geometry);


--
-- Name: waypoints_geom_idx; Type: INDEX; Schema: public; Owner: pmauduit; Tablespace: 
--

CREATE INDEX waypoints_geom_idx ON waypoints USING gist (wkb_geometry);



--
-- PostgreSQL database dump complete
--

style.mcss

Map {
  background-color: black;
}
#tracks {
  line-width: 0.7;
  line-color:green;
  line-opacity: 0.6;
  line-gamma: 0.8;
  line-comp-op: screen;
}