Skip to content

Instantly share code, notes, and snippets.

@bdon
Created March 27, 2014 16:21
Show Gist options
  • Save bdon/9811498 to your computer and use it in GitHub Desktop.
Save bdon/9811498 to your computer and use it in GitHub Desktop.
bdon@bdon-H87N-WIFI:~$ osm2pgsql -d osm --slim --flat-nodes flat-nodes.bin --number-processes 6 -C 2000 -H /tmp/ north-america-latest.osm.pbf
osm2pgsql SVN version 0.84.0 (64bit id space)
Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE: table "planet_osm_point" does not exist, skipping
NOTICE: table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE: table "planet_osm_line" does not exist, skipping
NOTICE: table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE: table "planet_osm_polygon" does not exist, skipping
NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE: table "planet_osm_roads" does not exist, skipping
NOTICE: table "planet_osm_roads_tmp" does not exist, skipping
Using built-in tag processing pipeline
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=2000MB, maxblocks=256000*8192, allocation method=11
Mid: loading persistent node cache from flat-nodes.bin
Allocated space for persistent node cache file
Maximum node in persistent node cache: 0
Mid: pgsql, scale=100 cache=2000
Setting up table: planet_osm_nodes
NOTICE: table "planet_osm_nodes" does not exist, skipping
Setting up table: planet_osm_ways
NOTICE: table "planet_osm_ways" does not exist, skipping
Setting up table: planet_osm_rels
NOTICE: table "planet_osm_rels" does not exist, skipping
Reading in file: north-america-latest.osm.pbf
Processing: Node(663342k 1666.7k/s) Way(43096k 18.03k/s) Relation(357270 42.04/s) parse time: 11286s
Node stats: total(663342331), max(2735040242) in 398s
Way stats: total(43096680), max(268148271) in 2390s
Relation stats: total(357271), max(3595053) in 8498s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads
Going over pending ways...
Maximum node in persistent node cache: 2735734783
20498559 ways are pending
Using 6 helper-processes
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Helper process 5 out of 6 initialised
Helper process 2 out of 6 initialised
Helper process 0 out of 6 initialised
Helper process 4 out of 6 initialised
Helper process 3 out of 6 initialised
Helper process 1 out of 6 initialised
Process 4 finished processing 3416426 ways in 5911 sec
Process 5 finished processing 3416426 ways in 5911 sec
Process 3 finished processing 3416426 ways in 5911 sec
Process 0 finished processing 3416427 ways in 5911 sec
Process 1 finished processing 3416427 ways in 5911 sec
Process 2 finished processing 3416427 ways in 5911 sec
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
All child processes exited
20498559 Pending ways took 5913s at a rate of 3466.69/s
Committing transaction for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_polygon
Committing transaction for planet_osm_roads
Going over pending relations...
Maximum node in persistent node cache: 2735734783
0 relations are pending
Using 6 helper-processes
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Mid: loading persistent node cache from flat-nodes.bin
Maximum node in persistent node cache: 2735734783
Process 3 finished processing 0 relations in 2 sec
Process 4 finished processing 0 relations in 2 sec
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Process 0 finished processing 0 relations in 3 sec
Process 2 finished processing 0 relations in 3 sec
Process 1 finished processing 0 relations in 3 sec
Process 5 finished processing 0 relations in 3 sec
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
Maximum node in persistent node cache: 2735734783
All child processes exited
0 Pending relations took 3s at a rate of 0.00/s
Sorting data and creating indexes for planet_osm_point
node cache: stored: 229125070(34.54%), storage efficiency: 87.40% (dense blocks: 217813, sparse nodes: 19551745), hit rate: 38.58%
Sorting data and creating indexes for planet_osm_line
Sorting data and creating indexes for planet_osm_polygon
Sorting data and creating indexes for planet_osm_roads
Maximum node in persistent node cache: 2735734783
Stopping table: planet_osm_nodes
Stopping table: planet_osm_ways
Stopped table: planet_osm_nodes in 0s
Building index on table: planet_osm_ways (fastupdate=off)
Stopping table: planet_osm_rels
Building index on table: planet_osm_rels (fastupdate=off)
Stopped table: planet_osm_rels in 202s
Analyzing planet_osm_roads finished
Analyzing planet_osm_point finished
Copying planet_osm_roads to cluster by geometry finished
Creating geometry index on planet_osm_roads
Creating osm_id index on planet_osm_roads
Creating indexes on planet_osm_roads finished
All indexes on planet_osm_roads created in 2346s
Completed planet_osm_roads
Copying planet_osm_point to cluster by geometry finished
Creating geometry index on planet_osm_point
Creating osm_id index on planet_osm_point
Creating indexes on planet_osm_point finished
All indexes on planet_osm_point created in 3055s
Completed planet_osm_point
Analyzing planet_osm_line finished
Analyzing planet_osm_polygon finished
Copying planet_osm_polygon to cluster by geometry finished
Creating geometry index on planet_osm_polygon
Creating osm_id index on planet_osm_polygon
Copying planet_osm_line to cluster by geometry finished
Creating geometry index on planet_osm_line
Creating indexes on planet_osm_polygon finished
All indexes on planet_osm_polygon created in 35633s
Completed planet_osm_polygon
Creating osm_id index on planet_osm_line
Creating indexes on planet_osm_line finished
All indexes on planet_osm_line created in 40444s
Completed planet_osm_line
Stopped table: planet_osm_ways in 87152s
Osm2pgsql took 104357s overall
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment