Skip to content

Instantly share code, notes, and snippets.

@mateusfmello
Created October 31, 2019 13:30
Show Gist options
  • Save mateusfmello/fa3eae11ddbb992e21c45ce88d697542 to your computer and use it in GitHub Desktop.
Save mateusfmello/fa3eae11ddbb992e21c45ce88d697542 to your computer and use it in GitHub Desktop.
Error import container
[root@localhost ~]# docker run -v /root/south-america-latest.osm.pbf:/data.osm.pbf -v /root/south-america.poly:/data.poly -v openstreetmap-data:/var/lib/postgresql/12/main -v openstreetmap-rendered-tiles:/var/lib/mod_tile overv/openstreetmap-tile-server import
Unable to find image 'overv/openstreetmap-tile-server:latest' locally
latest: Pulling from overv/openstreetmap-tile-server
35c102085707: Pull complete
251f5509d51d: Pull complete
8e829fe70a46: Pull complete
6001e1789921: Pull complete
10b3be299f3e: Pull complete
03cb58c3f4d8: Pull complete
f6ce5baef70c: Pull complete
ac1080c85a85: Pull complete
2e8367d63db9: Pull complete
a4500d2d28a8: Pull complete
cf5024d26150: Pull complete
7c4b5e261cfd: Pull complete
9dd1ff037417: Pull complete
c87158b93cd8: Pull complete
dd8b1ce4f9cb: Pull complete
12e1a915c466: Pull complete
2d25da130c2c: Pull complete
021e2a0018fd: Pull complete
bdbdb8295bbb: Pull complete
16692e610130: Pull complete
e6d5182fb3a5: Pull complete
18c88496838d: Pull complete
7ca3eb3f3a85: Pull complete
3f77a5855261: Pull complete
94d33ab1c047: Pull complete
0ae9921c1cce: Pull complete
4232a78c5123: Pull complete
c7750d40cb8a: Pull complete
0018297beab0: Pull complete
9610de221906: Pull complete
a05c79d9bc10: Pull complete
a4a4c6c1eec6: Pull complete
01b602f13d44: Pull complete
0040daf6dfb4: Pull complete
0106b76481fc: Pull complete
0acafd90377c: Pull complete
4eacac0cce06: Pull complete
Digest: sha256:ddfec5fc85e2dab2e572fc83bb3dd0ca4f189574110b731f99beef8f4916b5b4
Status: Downloaded newer image for overv/openstreetmap-tile-server:latest
+ '[' 1 -ne 1 ']'
+ '[' import = import ']'
+ createPostgresConfig
+ cp /etc/postgresql/12/main/postgresql.custom.conf.tmpl /etc/postgresql/12/main/postgresql.custom.conf
+ sudo -u postgres echo 'autovacuum = on'
+ cat /etc/postgresql/12/main/postgresql.custom.conf
# Suggested minimal settings from
# https://ircama.github.io/osm-carto-tutorials/tile-server-ubuntu/
shared_buffers = 128MB
min_wal_size = 1GB
max_wal_size = 2GB
maintenance_work_mem = 256MB
# Suggested settings from
# https://github.com/openstreetmap/chef/blob/master/roles/tile.rb#L38-L45
max_connections = 250
temp_buffers = 32MB
work_mem = 128MB
wal_buffers = 1024kB
wal_writer_delay = 500ms
commit_delay = 10000
# checkpoint_segments = 60 # unrecognized in psql 10.7.1
max_wal_size = 2880MB
random_page_cost = 1.1
track_activity_query_size = 16384
autovacuum_vacuum_scale_factor = 0.05
autovacuum_analyze_scale_factor = 0.02
listen_addresses = '*'
autovacuum = on
+ service postgresql start
* Starting PostgreSQL 12 database server
...done.
+ sudo -u postgres createuser renderer
+ sudo -u postgres createdb -E UTF8 -O renderer gis
+ sudo -u postgres psql -d gis -c 'CREATE EXTENSION postgis;'
CREATE EXTENSION
+ sudo -u postgres psql -d gis -c 'CREATE EXTENSION hstore;'
CREATE EXTENSION
+ sudo -u postgres psql -d gis -c 'ALTER TABLE geometry_columns OWNER TO renderer;'
ALTER TABLE
+ sudo -u postgres psql -d gis -c 'ALTER TABLE spatial_ref_sys OWNER TO renderer;'
ALTER TABLE
+ setPostgresPassword
+ sudo -u postgres psql -c 'ALTER USER renderer PASSWORD '\''renderer'\'''
ALTER ROLE
+ '[' '!' -f /data.osm.pbf ']'
+ osmium fileinfo /data.osm.pbf
+ osmium fileinfo /data.osm.pbf
+ grep osmosis_replication_timestamp=
+ cut -b35-44
++ cat /var/lib/mod_tile/replication_timestamp.txt
+ REPLICATION_TIMESTAMP=2019-10-30
+ sudo -u renderer openstreetmap-tiles-update-expire 2019-10-30
--2019-10-31 10:55:15-- http://replicate-sequences.osm.mazdermind.de/?2019-10-30T00:00:00Z
Resolving replicate-sequences.osm.mazdermind.de (replicate-sequences.osm.mazdermind.de)... 188.68.50.52, 2a03:4000:6:d080::1
Connecting to replicate-sequences.osm.mazdermind.de (replicate-sequences.osm.mazdermind.de)|188.68.50.52|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://replicate-sequences.osm.mazdermind.de/?2019-10-30T00:00:00Z [following]
--2019-10-31 10:55:16-- https://replicate-sequences.osm.mazdermind.de/?2019-10-30T00:00:00Z
Connecting to replicate-sequences.osm.mazdermind.de (replicate-sequences.osm.mazdermind.de)|188.68.50.52|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 332 [text/plain]
Saving to: '/var/lib/mod_tile/.osmosis/state.txt'
0K 100% 42.9M=0s
2019-10-31 10:55:18 (42.9 MB/s) - '/var/lib/mod_tile/.osmosis/state.txt' saved [332/332]
+ '[' -f /data.poly ']'
+ sudo -u renderer cp /data.poly /var/lib/mod_tile/data.poly
+ sudo -u renderer osm2pgsql -d gis --create --slim -G --hstore --tag-transform-script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua --number-processes 4 -S /home/renderer/src/openstreetmap-carto/openstreetmap-carto.style /data.osm.pbf
osm2pgsql version 1.0.0 (64 bit id space)
Allocating memory for dense node cache
Allocating dense node cache in one big chunk
Allocating memory for sparse node cache
Sharing dense sparse
Node-cache: cache=800MB, maxblocks=12800*65536, allocation method=11
Mid: pgsql, cache=800
Setting up table: planet_osm_nodes
Setting up table: planet_osm_ways
Setting up table: planet_osm_rels
Using lua based tag processing pipeline with script /home/renderer/src/openstreetmap-carto/openstreetmap-carto.lua
Using projection SRS 3857 (Spherical Mercator)
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Reading in file: /data.osm.pbf
Using PBF parser.
Processing: Node(255636k 268.2k/s) Way(21770k 4.14k/s) Relation(320860 229.84/s) parse time: 7604s
Node stats: total(255636195), max(6933104389) in 953s
Way stats: total(21770464), max(740526458) in 5255s
Relation stats: total(321227), max(10237540) in 1396s
Sorting data and creating indexes for planet_osm_point
Sorting data and creating indexes for planet_osm_line
Sorting data and creating indexes for planet_osm_polygon
Sorting data and creating indexes for planet_osm_roads
Copying planet_osm_point to cluster by geometry finished
Creating geometry index on planet_osm_point
Creating osm_id index on planet_osm_point
Creating indexes on planet_osm_point finished
All indexes on planet_osm_point created in 504s
Completed planet_osm_point
Stopping table: planet_osm_nodes
Stopped table: planet_osm_nodes in 0s
Stopping table: planet_osm_ways
Building index on table: planet_osm_ways
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
Stopping table: planet_osm_rels
Connection to database failed: FATAL: the database system is in recovery mode
Error occurred, cleaning up
+ sudo -u postgres psql -d gis -f indexes.sql
psql: error: could not connect to server: FATAL: the database system is in recovery mode
+ touch /var/lib/mod_tile/planet-import-complete
+ service postgresql stop
* Stopping PostgreSQL 12 database server
...done.
+ exit 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment