Instantly share code, notes, and snippets.

Embed
What would you like to do?

Ultimate Goal: valhalla running on AWS ECS, behind a load balancer, in multiple regions, with auto scaling, with data updating once a week.

Diary

First goal: Build with docker, then cut tiles from a small extract and get server to run locally.

jesse:projects/ $ mkdir ~/valhalla-data
jesse:projects/ $ cd ~/valhalla-data
jesse:projects/ $ wget http://download.geofabrik.de/north-america/us/california-latest.osm.pbf
[output abbreviated]
jesse:projects/ $ cd ~/projects
jesse:projects/ $ git clone git@github.com:valhalla/docker.git
[output abbreviated]
jesse:projects/ $ mv docker valhalla-docker/
jesse:projects/ $ cd valhalla-docker
jesse:valhalla-docker/ (master) $ docker build -t valhalla -f Dockerfile-ppa .
[output abbreviated]
jesse:valhalla-docker/ (master) $ docker run --rm -it -v /Users/jesse/valhalla-data:/data/valhalla/:delegated valhalla /scripts/cut_tiles.sh
[output abbreviated]
jesse:valhalla-docker/ (master) $ docker run --rm -it -p 8002:8002 -v /Users/jesse/valhalla-data/tiles/:/data/valhalla/:delegated valhalla
[output abbreviated]

It works. I can generate routes in California.

Next goal: Generate tiles for north america to get an idea how long it will take to do the planet.

wget http://download.geofabrik.de/north-america-latest.osm.pbf

docker run --rm -it -v /Users/jesse/valhalla-data-north-america:/data/valhalla/:delegated valhalla /scripts/cut_tiles.sh

[INFO] updating config.
[INFO] building admins.
2018/01/04 21:01:45.784921 [INFO] Parsing files: /data/valhalla/extracts/north-america-latest.osm.pbf
2018/01/04 21:01:45.785713 [INFO] Parsing relations...
2018/01/04 21:04:37.434146 [INFO] Finished with 114 admin polygons comprised of 17091 ways
2018/01/04 21:04:37.434184 [INFO] Parsing ways...
2018/01/04 21:08:26.922304 [INFO] Finished with 7261 ways comprised of 892134 nodes
2018/01/04 21:08:26.922340 [INFO] Parsing nodes...
2018/01/04 21:10:02.007894 [INFO] Processed 500000 nodes on ways
2018/01/04 21:11:52.406297 [INFO] Finished with 884768 nodes
2018/01/04 21:12:12.379805 [INFO] Created admin access table.
2018/01/04 21:12:12.424873 [INFO] Created admin table.
2018/01/04 21:12:29.160665 [INFO] Inserted 98 admin areas
2018/01/04 21:12:29.763053 [INFO] Created spatial index
2018/01/04 21:12:29.766743 [INFO] Created Level index
2018/01/04 21:12:29.771034 [INFO] Created Drive On Right index
2018/01/04 21:12:29.798275 [INFO] Done updating drive on right column.
2018/01/04 21:12:48.068164 [INFO] Done updating Parent admin
2018/01/04 21:12:48.069327 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069393 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069482 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069571 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069629 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069705 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069748 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069829 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069896 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.069959 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070024 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070072 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070164 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070249 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070305 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070373 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070427 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070471 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070537 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.070608 [ERROR] sqlite3_step() error: NOT NULL constraint failed: admin_access.admin_id.  Ignore if not using a planet extract.
2018/01/04 21:12:48.072202 [INFO] Finished.
[INFO] building timezones.
--2018-01-04 21:12:48--  http://efele.net/maps/tz/world/tz_world_mp.zip
Resolving efele.net (efele.net)... 216.250.120.67
Connecting to efele.net (efele.net)|216.250.120.67|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15619058 (15M) [application/zip]
Saving to: 'tz_world_mp.zip'

100%[======================================================================================================================================================>] 15,619,058  2.31MB/s   in 6.7s

2018-01-04 21:12:55 (2.21 MB/s) - 'tz_world_mp.zip' saved [15619058/15619058]

Archive:  ./tz_world_mp.zip
  inflating: world/index.html
  inflating: world/tz_world.png
  inflating: world/tz_world_mp.prj
  inflating: world/tz_world_mp.shp
  inflating: world/tz_world_mp.shx
  inflating: world/tz_world_mp.dbf
SQLite version: 3.8.2
SpatiaLite version: 4.1.1
Inserted 418 rows into 'tz_world' from './world/tz_world_mp.shp'
SpatiaLite version ..: 4.1.1    Supported Extensions:
    - 'VirtualShape'    [direct Shapefile access]
    - 'VirtualDbf'      [direct DBF access]
    - 'VirtualXL'       [direct XLS access]
    - 'VirtualText'     [direct CSV/TXT access]
    - 'VirtualNetwork'  [Dijkstra shortest path]
    - 'RTree'       [Spatial Index - R*Tree]
    - 'MbrCache'        [Spatial Index - MBR cache]
    - 'VirtualSpatialIndex' [R*Tree metahandler]
    - 'VirtualXPath'    [XML Path Language - XPath]
    - 'VirtualFDO'      [FDO-OGR interoperability]
    - 'SpatiaLite'      [Spatial SQL - OGC]
PROJ.4 version ......: Rel. 4.8.0, 6 March 2012
GEOS version ........: 3.4.2-CAPI-1.8.2 r3921
1
[INFO] building tiles.
2018/01/04 21:13:03.451409 [INFO] Parsing files: /data/valhalla/extracts/north-america-latest.osm.pbf
2018/01/04 21:13:03.451760 [INFO] Parsing ways...
2018/01/04 21:49:48.789264 [INFO] Finished with 24247245 routable ways containing 293357288 nodes
2018/01/04 21:49:48.789293 [INFO] Sorting osm access tags by way id...
2018/01/04 21:49:48.793204 [INFO] Parsing relations...
2018/01/04 21:55:16.863724 [INFO] Finished with 144022 simple restrictions
2018/01/04 21:55:16.863760 [INFO] Finished with 1 lane connections
2018/01/04 21:55:17.127937 [INFO] Sorting complex restrictions by from id...
2018/01/04 21:55:17.177892 [INFO] Sorting osm way node references by node id...
2018/01/04 22:08:24.066687 [INFO] Finished
2018/01/04 22:08:24.177582 [INFO] Parsing nodes...
2018/01/04 22:32:34.302836 [INFO] Finished with 256678972 nodes contained in routable ways
2018/01/04 22:32:34.302882 [INFO] Sorting osm way node references by way index and node shape index...
2018/01/04 22:44:26.909201 [INFO] Finished at changeset id 55112826
2018/01/04 22:44:28.581123 [INFO] Creating graph edges from ways...
2018/01/05 03:49:14.473741 [INFO] Finished with 54707844 graph edges
2018/01/05 03:49:14.596740 [INFO] Sorting graph...
2018/01/05 03:54:36.560804 [INFO] Finished with 42276773 graph nodes
2018/01/05 03:54:37.030277 [INFO] Reclassifying link graph edges...
2018/01/05 03:54:45.318365 [INFO] Class: 0 exit count = 67162
2018/01/05 03:54:45.318437 [INFO] Class: 1 exit count = 52228
2018/01/05 03:54:45.318441 [INFO] Class: 2 exit count = 75797
2018/01/05 03:54:45.318443 [INFO] Class: 3 exit count = 102438
2018/01/05 03:54:45.318445 [INFO] Class: 4 exit count = 49774
2018/01/05 03:54:45.318457 [INFO] Class: 5 exit count = 5005
2018/01/05 03:54:45.318459 [INFO] Class: 6 exit count = 20909
2018/01/05 03:54:45.318461 [INFO] Class: 7 exit count = 0
2018/01/05 03:55:43.160586 [INFO] Finished with 315295 reclassified.  Turn channel count = 150529
2018/01/05 03:55:43.419628 [INFO] Reclassifying ferry connection graph edges...
2018/01/05 03:56:03.401034 [INFO] Finished ReclassifyFerryEdges: ferry_endpoint_count = 366, 9694 edges reclassified.
2018/01/05 03:56:03.683209 [INFO] Building 25726 tiles with 4 threads...
2018/01/05 04:06:26.537356 [WARN] Restrictions mask exceeds allowable limit on wayid: 54293033
2018/01/05 04:06:26.542225 [WARN] Restrictions mask exceeds allowable limit: 507
2018/01/05 04:08:03.303089 [WARN] Too many names for edgeindex: 46531679
2018/01/05 04:13:09.477822 [INFO] Finished
2018/01/05 04:16:45.757401 [INFO] Enhancing local graph...
2018/01/05 04:19:26.237059 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:19:26.373839 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:24:33.704801 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:24:33.951747 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.420307 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.467240 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.467306 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.472742 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.484786 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:31:55.484889 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:01.004303 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:02.533913 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:02.534080 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:03.504844 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:03.504978 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 04:37:03.504996 [WARN] Exceeding maximum.  Average speed: 141
2018/01/05 05:02:36.522653 [INFO] Finished with max_density 29.894590 and unreachable 276842
2018/01/05 05:02:36.529320 [INFO] Transit directory not found. Transit will not be added.
2018/01/05 05:02:36.539722 [INFO] HierarchyBuilder
2018/01/05 05:26:22.681717 [INFO] Done HierarchyBuilder
2018/01/05 05:26:23.215178 [INFO] Creating shortcuts on level 1
2018/01/05 05:26:49.157892 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:27:00.081742 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:27:44.348075 [INFO] Finished with 1706634 shortcuts
2018/01/05 05:27:44.348104 [INFO] Creating shortcuts on level 0
2018/01/05 05:27:46.794444 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:27:46.795300 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:27:46.795325 [WARN] Exceeding max shortcut edges from a node: 9
2018/01/05 05:27:46.795339 [WARN] Exceeding max shortcut edges from a node: 10
2018/01/05 05:27:46.795342 [WARN] Exceeding max shortcut edges from a node: 11
2018/01/05 05:27:46.795372 [WARN] Exceeding max shortcut edges from a node: 12
2018/01/05 05:27:46.795408 [WARN] Exceeding max shortcut edges from a node: 13
2018/01/05 05:27:46.795412 [WARN] Exceeding max shortcut edges from a node: 14
2018/01/05 05:27:46.795874 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:27:46.797048 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:01.965938 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:04.809473 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:04.811942 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:04.992362 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:04.992516 [WARN] Exceeding max shortcut edges from a node: 9
2018/01/05 05:28:04.992538 [WARN] Exceeding max shortcut edges from a node: 10
2018/01/05 05:28:04.992597 [WARN] Exceeding max shortcut edges from a node: 11
2018/01/05 05:28:04.992622 [WARN] Exceeding max shortcut edges from a node: 12
2018/01/05 05:28:04.992642 [WARN] Exceeding max shortcut edges from a node: 13
2018/01/05 05:28:04.992671 [WARN] Exceeding max shortcut edges from a node: 14
2018/01/05 05:28:04.992766 [WARN] Exceeding max shortcut edges from a node: 15
2018/01/05 05:28:04.998579 [WARN] Exceeding max shortcut edges from a node: 8
2018/01/05 05:28:08.752475 [INFO] Finished with 582734 shortcuts
2018/01/05 05:31:23.984320 [INFO] Adding Restrictions at level 2
2018/01/05 05:35:16.622886 [INFO] --Forward restrictions added: 791
2018/01/05 05:35:16.622935 [INFO] --Reverse restrictions added: 535
2018/01/05 05:35:33.202821 [INFO] Adding Restrictions at level 1
2018/01/05 05:36:10.996029 [INFO] --Forward restrictions added: 5494
2018/01/05 05:36:10.996085 [INFO] --Reverse restrictions added: 5624
2018/01/05 05:36:11.931544 [INFO] Adding Restrictions at level 0
2018/01/05 05:36:26.303532 [INFO] --Forward restrictions added: 5931
2018/01/05 05:36:26.303575 [INFO] --Reverse restrictions added: 6049
2018/01/05 05:36:26.303616 [INFO] Finished
2018/01/05 05:44:07.374200 [INFO] Validating, finishing and binning tiles...
2018/01/05 06:01:14.764533 [INFO] Finished
2018/01/05 06:01:14.764581 [INFO] Binning inter-tile edges...
2018/01/05 06:03:24.913201 [INFO] Finished
2018/01/05 06:03:24.966142 [WARN] Possible duplicates at level: 0 = 128
2018/01/05 06:03:24.966192 [WARN] Possible duplicates at level: 1 = 464
2018/01/05 06:03:24.966197 [WARN] Possible duplicates at level: 2 = 1936
/data/valhalla/planet_2018_01_04-21_01_45 /data/valhalla
[INFO] building connectivity.
2018/01/05 06:03:30.295066 [WARN] Tile extract could not be loaded
[INFO] building stats.
2018/01/05 06:03:38.243690 [WARN] Tile extract could not be loaded
2018/01/05 06:11:13.632724 [INFO] Gathering information about the tiles in /data/valhalla/tiles
2018/01/05 06:11:42.598935 [INFO] Finished
2018/01/05 06:11:42.628666 [INFO] Writing statistics database
2018/01/05 06:11:42.628720 [INFO] Creating tables
2018/01/05 06:12:03.598798 [INFO] Created tile tables
2018/01/05 06:12:03.610561 [INFO] Created country tables
2018/01/05 06:12:03.623179 [INFO] Created exit tables
2018/01/05 06:12:07.083931 [INFO] Tile info inserted
2018/01/05 06:12:07.096603 [INFO] Country info inserted
2018/01/05 06:12:07.109862 [INFO] Exit info inserted
2018/01/05 06:12:16.117532 [INFO] Statistics database saved to statistics.sqlite
2018/01/05 06:12:16.188547 [INFO] MapRoulette tasks saved to maproulette_tasks.geojson
2018/01/05 06:12:16.189167 [INFO] 5085 tasks generated
[INFO] exporting edges.
2018/01/05 06:12:17.110972 [WARN] Tile extract could not be loaded
2018/01/05 06:12:17.111093 [INFO] Enumerating edges...
2018/01/05 06:16:22.931411 [INFO] Exporting 124565194 edges
2018/01/05 06:16:22.932296 [INFO] 0%
2018/01/05 06:16:39.566522 [INFO] 1%
2018/01/05 06:16:50.133371 [INFO] 2%
2018/01/05 06:17:02.572066 [INFO] 3%
2018/01/05 06:17:16.052838 [INFO] 4%
2018/01/05 06:17:28.297741 [INFO] 5%
2018/01/05 06:17:38.109070 [INFO] 6%
2018/01/05 06:17:49.598291 [INFO] 7%
2018/01/05 06:18:03.824960 [INFO] 8%
2018/01/05 06:18:14.814980 [INFO] 9%
2018/01/05 06:18:26.593283 [INFO] 10%
2018/01/05 06:18:38.874514 [INFO] 11%
2018/01/05 06:18:51.733134 [INFO] 12%
2018/01/05 06:19:04.734742 [INFO] 13%
2018/01/05 06:19:15.425781 [INFO] 14%
2018/01/05 06:19:25.127380 [INFO] 15%
2018/01/05 06:19:37.522647 [INFO] 16%
2018/01/05 06:19:47.430317 [INFO] 17%
2018/01/05 06:19:59.279151 [INFO] 18%
2018/01/05 06:20:08.581754 [INFO] 19%
2018/01/05 06:20:19.315003 [INFO] 20%
2018/01/05 06:20:32.707481 [INFO] 21%
2018/01/05 06:20:42.209242 [INFO] 22%
2018/01/05 06:20:55.244519 [INFO] 23%
2018/01/05 06:21:06.165215 [INFO] 24%
2018/01/05 06:21:18.425541 [INFO] 25%
2018/01/05 06:21:31.237053 [INFO] 26%
2018/01/05 06:21:44.763570 [INFO] 27%
2018/01/05 06:21:56.250720 [INFO] 28%
2018/01/05 06:22:09.477691 [INFO] 29%
2018/01/05 06:22:27.038069 [INFO] 30%
2018/01/05 06:22:39.919188 [INFO] 31%
2018/01/05 06:22:50.050785 [INFO] 32%
2018/01/05 06:23:01.985587 [INFO] 33%
2018/01/05 06:23:12.835074 [INFO] 34%
2018/01/05 06:23:22.692215 [INFO] 35%
2018/01/05 06:23:34.624311 [INFO] 36%
2018/01/05 06:23:44.574122 [INFO] 37%
2018/01/05 06:23:54.532581 [INFO] 38%
2018/01/05 06:24:05.687952 [INFO] 39%
2018/01/05 06:24:16.839960 [INFO] 40%
2018/01/05 06:24:29.019617 [INFO] 41%
2018/01/05 06:24:41.362306 [INFO] 42%
2018/01/05 06:24:54.446258 [INFO] 43%
2018/01/05 06:25:05.234053 [INFO] 44%
2018/01/05 06:25:15.800155 [INFO] 45%
2018/01/05 06:25:28.505557 [INFO] 46%
2018/01/05 06:25:40.877188 [INFO] 47%
2018/01/05 06:25:55.341354 [INFO] 48%
2018/01/05 06:26:14.029497 [INFO] 49%
2018/01/05 06:26:28.436472 [INFO] 50%
2018/01/05 06:26:38.830643 [INFO] 51%
2018/01/05 06:26:50.900954 [INFO] 52%
2018/01/05 06:27:07.140562 [INFO] 53%
2018/01/05 06:27:18.795451 [INFO] 54%
2018/01/05 06:27:35.137059 [INFO] 55%
2018/01/05 06:27:54.178127 [INFO] 56%
2018/01/05 06:28:10.047952 [INFO] 57%
2018/01/05 06:28:26.980780 [INFO] 58%
2018/01/05 06:28:42.635282 [INFO] 59%
2018/01/05 06:28:56.587167 [INFO] 60%
2018/01/05 06:29:11.855981 [INFO] 61%
2018/01/05 06:29:24.100461 [INFO] 62%
2018/01/05 06:29:38.704400 [INFO] 63%
2018/01/05 06:29:50.750824 [INFO] 64%
2018/01/05 06:30:05.757391 [INFO] 65%
2018/01/05 06:30:18.855064 [INFO] 66%
2018/01/05 06:30:36.528800 [INFO] 67%
2018/01/05 06:30:52.782027 [INFO] 68%
2018/01/05 06:31:06.434348 [INFO] 69%
2018/01/05 06:31:20.800738 [INFO] 70%
2018/01/05 06:31:37.208237 [INFO] 71%
2018/01/05 06:31:55.816977 [INFO] 72%
2018/01/05 06:32:09.912655 [INFO] 73%
2018/01/05 06:32:18.061723 [INFO] 74%
2018/01/05 06:32:26.917271 [INFO] 75%
2018/01/05 06:32:33.401168 [INFO] 76%
2018/01/05 06:32:44.526246 [INFO] 77%
2018/01/05 06:32:48.983831 [INFO] 78%
2018/01/05 06:32:55.579919 [INFO] 79%
2018/01/05 06:33:02.283279 [INFO] 80%
2018/01/05 06:33:08.215616 [INFO] 81%
2018/01/05 06:33:15.087624 [INFO] 82%
2018/01/05 06:33:21.580961 [INFO] 83%
2018/01/05 06:33:29.554751 [INFO] 84%
2018/01/05 06:33:37.057813 [INFO] 85%
2018/01/05 06:33:43.905701 [INFO] 86%
2018/01/05 06:33:50.471478 [INFO] 87%
2018/01/05 06:33:56.349098 [INFO] 88%
2018/01/05 06:34:02.906542 [INFO] 89%
2018/01/05 06:34:08.937036 [INFO] 90%
2018/01/05 06:34:14.378459 [INFO] 91%
2018/01/05 06:34:21.086969 [INFO] 92%
2018/01/05 06:34:27.287571 [INFO] 93%
2018/01/05 06:34:32.826415 [INFO] 94%
2018/01/05 06:34:40.211334 [INFO] 95%
2018/01/05 06:34:48.591187 [INFO] 96%
2018/01/05 06:34:54.656866 [INFO] 97%
2018/01/05 06:35:00.302420 [INFO] 98%
2018/01/05 06:35:06.779912 [INFO] 99%
2018/01/05 06:35:11.611516 [INFO] 100%
2018/01/05 06:35:11.611649 [INFO] Done
/data/valhalla/tiles /data/valhalla/planet_2018_01_04-21_01_45 /data/valhalla
mv: cannot move '/data/valhalla/tiles/2' to '/data/valhalla/planet_2018_01_04-21_01_45/2': Input/output error

It took 9 hours, made 21350mb of files, and failed on the very last step. Not sure what caused that error, but it seems like the tiles are all there, so i'll just move them manually.

$ mv tiles/2 north_america_2018_01_04/
$ du -ms valhalla-data-north-america/north_america_2018_01_04
21350   valhalla-data-north-america/north_america_2018_01_04

Run server to make sure those tiles work

docker run --rm -it -p 8002:8002 -v /Users/jesse/valhalla-data-north-america/planet_2018_01_04-21_01_45/:/data/valhalla/:delegated valhalla

2018-01-05 17:30:49,417 CRIT Supervisor running as root (no user in config file)
2018/01/05 17:30:50.526609 [ERROR] bad upstream endpoint
2018/01/05 17:30:50.526736 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/05 17:30:50.581547 [WARN] Tile extract could not be loaded
2018/01/05 17:30:50.581706 [WARN] Tile extract could not be loaded
2018/01/05 17:30:51.589880 [ERROR] bad upstream endpoint
2018/01/05 17:30:51.590385 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/05 17:30:53.604337 [ERROR] bad upstream endpoint
2018/01/05 17:30:53.604397 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/05 17:30:56.616593 [ERROR] bad upstream endpoint
2018/01/05 17:30:56.617057 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
0 2018/01/05 18:20:25.568723 GET /route?json={%22locations%22:[{%22lat%22:46.965259400349275,%22lon%22:-114.093017578125,%22type%22:%22break%22},{%22lat%22:46.7248003746672,%22lon%22:-113.51074218749999,%22type%22:%22break%22}],%22costing%22:%22pedestrian%22}&api_key=mapzen-U9QgQ9L HTTP/1.1
2018/01/05 18:20:25.568952 [INFO] Got Loki Request 0
2018/01/05 18:20:25.569754 [ANALYTICS] locations_count::2
2018/01/05 18:20:25.569780 [ANALYTICS] costing_type::pedestrian
2018/01/05 18:20:25.569883 [ANALYTICS] location_distance::51.788406km
2018/01/05 18:20:25.610154 [INFO] Got Thor Request 0
2018/01/05 18:20:25.610581 [ANALYTICS] travel_mode::1
2018/01/05 18:20:25.707482 [ANALYTICS] admin_state_iso::
2018/01/05 18:20:25.707507 [ANALYTICS] admin_country_iso::
2018/01/05 18:20:25.708542 [INFO] Got Odin Request 0
2018/01/05 18:20:25.709428 [INFO] trip_path_->node_size()=237
2018/01/05 18:20:25.945539 [INFO] maneuver_count::56
0 2018/01/05 18:20:25.947139 200 29421

They work!

Next goal: Get tile loading from s3 working.

The docs mention that it should be possible to store tiles on s3 and load them on demand, but i cant find anything more about it in the docs. Time to search the source. The conf file in the docker container has tile_dir, so i'll download the source and start search from there.

jesse:projects/ $ git clone git@github.com:valhalla/valhalla.git
Cloning into 'valhalla'...
remote: Counting objects: 58003, done.
remote: Compressing objects: 100% (56/56), done.
remote: Total 58003 (delta 26), reused 4 (delta 1), pack-reused 57946
Receiving objects: 100% (58003/58003), 83.84 MiB | 652.00 KiB/s, done.
Resolving deltas: 100% (36889/36889), done.
jesse:projects/ $ cd valhalla
jesse:valhalla/ (master) $ git submodule init
Submodule 'OSM-binary' (https://github.com/scrosby/OSM-binary.git) registered for path 'OSM-binary'
Submodule 'proto' (https://github.com/valhalla/osmlr-tile-spec.git) registered for path 'OSMLR'
Submodule 'rapidjson' (https://github.com/miloyip/rapidjson) registered for path 'rapidjson'
jesse:valhalla/ (master) $ git submodule update
Cloning into '/Users/jesse/projects/valhalla/OSM-binary'...
Cloning into '/Users/jesse/projects/valhalla/OSMLR'...
Cloning into '/Users/jesse/projects/valhalla/rapidjson'...
Submodule path 'OSM-binary': checked out '4e32fa236e919be5bb8dbb323437f026f24e5853'
Submodule path 'OSMLR': checked out 'ae07abeb7df49c18f7d94107259e4c27a02c315c'
Submodule path 'rapidjson': checked out 'f54b0e47a08782a6131cc3d60f94d038fa6e0a51'
jesse:valhalla/ (master) $ grep -r tile_dir -C 3 .
./Makefile.am-if DATA_TOOLS
./Makefile.am-#some of the tests want tiles of a certain location so lets build those
./Makefile.am-test/data/utrecht_tiles: valhalla_build_tiles
./Makefile.am:  @echo "Building Utrecht Tiles..." && rm -rf test/data/*utrecht_tiles && ./valhalla_build_tiles --inline-config '{"mjolnir":{"tile_dir":"test/data/tmp_utrecht_tiles","concurrency":1,"logging":{"type":""}}}' test/data/utrecht_netherlands.osm.pbf && mv test/data/tmp_utrecht_tiles test/data/utrecht_tiles
./Makefile.am-TEST_CLEANFILES += test/data/utrecht_tiles
./Makefile.am-
./Makefile.am-check_PROGRAMS += \
--
--
./scripts/valhalla_build_config-  'mjolnir': {
./scripts/valhalla_build_config-    'max_cache_size': 1000000000,
./scripts/valhalla_build_config-    'tile_url': None,
./scripts/valhalla_build_config:    'tile_dir': '/data/valhalla',
./scripts/valhalla_build_config-    'tile_extract': '/data/valhalla/tiles.tar',
./scripts/valhalla_build_config-    'admin': '/data/valhalla/admin.sqlite',
./scripts/valhalla_build_config-    'timezone': '/data/valhalla/tz_world.sqlite',
--
[output abbreviated]

tile_url, that sounds promising.

jesse:valhalla/ (master) $ grep -r tile_url -C 3 .
./scripts/valhalla_build_config-config = {
./scripts/valhalla_build_config-  'mjolnir': {
./scripts/valhalla_build_config-    'max_cache_size': 1000000000,
./scripts/valhalla_build_config:    'tile_url': None,
./scripts/valhalla_build_config-    'tile_dir': '/data/valhalla',
./scripts/valhalla_build_config-    'tile_extract': '/data/valhalla/tiles.tar',
./scripts/valhalla_build_config-    'admin': '/data/valhalla/admin.sqlite',
--
--
./scripts/valhalla_build_config-help_text = {
./scripts/valhalla_build_config-  'mjolnir': {
./scripts/valhalla_build_config-    'max_cache_size': 'Number of bytes per thread used to store tile data in memory',
./scripts/valhalla_build_config:    'tile_url': 'Location to read tiles from if they are not found in the tile_dir',
./scripts/valhalla_build_config-    'tile_dir': 'Location to read/write tiles to/from',
./scripts/valhalla_build_config-    'tile_extract': 'Location to read tiles from tar',
./scripts/valhalla_build_config-    'admin': 'Location of sqlite file holding admin polygons created with valhalla_build_admins',
--
--
./src/baldr/graphreader.cc-
./src/baldr/graphreader.cc-// Constructor using separate tile files
./src/baldr/graphreader.cc-GraphReader::GraphReader(const boost::property_tree::ptree& pt)
./src/baldr/graphreader.cc:    : tile_url_(pt.get<std::string>("tile_url", "")),
./src/baldr/graphreader.cc-      tile_dir_(pt.get<std::string>("tile_dir")),
./src/baldr/graphreader.cc-      tile_extract_(get_extract_instance(pt)),
./src/baldr/graphreader.cc-      cache_(TileCacheFactory::createTileCache(pt)) {
--
[output abbreviated]

It looks like this should be possible, so i'll make a new s3 bucket, and start those north american tiles uploading while i figure out how to configure it.

jesse:valhalla-data-north-america/ $ mv planet_2018_01_04-21_01_45 north_america_2018_01_04
jesse:valhalla-data-north-america/ $ s3cmd put north_america_2018_01_04 s3://com.gaiagps.routingdata/
ERROR: Parameter problem: Use --recursive to upload a directory: north_america_2018_01_04
jesse:valhalla-data-north-america/ $ s3cmd put --recursive north_america_2018_01_04 s3://com.gaiagps.routingdata/
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
upload: 'north_america_2018_01_04/0/002/359.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/359.gph'  [1 of 29157]
 195928 of 195928   100% in    1s   108.81 kB/s  done
upload: 'north_america_2018_01_04/0/002/359.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/359.gph'  [1 of 29157]
 195928 of 195928   100% in    1s   121.27 kB/s  done
upload: 'north_america_2018_01_04/0/002/360.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/360.gph'  [2 of 29157]
 1273880 of 1273880   100% in    2s   504.81 kB/s  done
upload: 'north_america_2018_01_04/0/002/361.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/361.gph'  [3 of 29157]
 1627264 of 1627264   100% in    2s   609.05 kB/s  done
upload: 'north_america_2018_01_04/0/002/362.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/362.gph'  [4 of 29157]
 183552 of 183552   100% in    0s   455.30 kB/s  done
upload: 'north_america_2018_01_04/0/002/435.gph' -> 's3://com.gaiagps.routingdata/north_america_2018_01_04/0/002/435.gph'  [5 of 29157]
 644504 of 644504   100% in    1s   571.86 kB/s  done

 [output abbreviated]

Now to look at graphreader.cc and figure out how to configure it. It doesn't look like it directly supports reading from s3, it just want's an http url. So i'll make that s3 bucket public and enable website hosting.

Now i'll try and launch the docker image, edit the config file, and start server.

jesse:valhalla-docker/ (master) $ docker run --rm -it -p 8002:8002 valhalla /bin/bash

root@5eefb4b4c473:/# nano /conf/valhalla.json
bash: nano: command not found
root@5eefb4b4c473:/# apt-get install nano
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package nano
root@5eefb4b4c473:/# apt-get update
Ign http://archive.ubuntu.com trusty InRelease
Get:1 http://ppa.launchpad.net trusty InRelease [16.0 kB]
Get:2 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB]
Get:3 http://archive.ubuntu.com trusty-security InRelease [65.9 kB]
Get:4 http://ppa.launchpad.net trusty InRelease [16.0 kB]
Get:5 http://archive.ubuntu.com trusty Release.gpg [933 B]
Get:6 http://archive.ubuntu.com trusty Release [58.5 kB]
Get:7 http://ppa.launchpad.net trusty/main amd64 Packages [869 B]
Get:8 http://archive.ubuntu.com trusty-updates/main Sources [505 kB]
Get:9 http://ppa.launchpad.net trusty/main amd64 Packages [22.4 kB]
Get:10 http://archive.ubuntu.com trusty-updates/restricted Sources [6449 B]
Get:11 http://archive.ubuntu.com trusty-updates/universe Sources [246 kB]
Get:12 http://archive.ubuntu.com trusty-updates/main amd64 Packages [1299 kB]
Get:13 http://archive.ubuntu.com trusty-updates/restricted amd64 Packages [21.4 kB]
Get:14 http://archive.ubuntu.com trusty-updates/universe amd64 Packages [564 kB]
Get:15 http://archive.ubuntu.com trusty-security/main Sources [188 kB]
Get:16 http://archive.ubuntu.com trusty-security/restricted Sources [5050 B]
Get:17 http://archive.ubuntu.com trusty-security/universe Sources [80.6 kB]
Get:18 http://archive.ubuntu.com trusty-security/main amd64 Packages [865 kB]
Get:19 http://archive.ubuntu.com trusty-security/restricted amd64 Packages [18.0 kB]
Get:20 http://archive.ubuntu.com trusty-security/universe amd64 Packages [253 kB]
Get:21 http://archive.ubuntu.com trusty/main Sources [1335 kB]
Get:22 http://archive.ubuntu.com trusty/restricted Sources [5335 B]
Get:23 http://archive.ubuntu.com trusty/universe Sources [7926 kB]
Get:24 http://archive.ubuntu.com trusty/main amd64 Packages [1743 kB]
Get:25 http://archive.ubuntu.com trusty/restricted amd64 Packages [16.0 kB]
Get:26 http://archive.ubuntu.com trusty/universe amd64 Packages [7589 kB]
Fetched 22.9 MB in 20s (1129 kB/s)
Reading package lists... Done
root@5eefb4b4c473:/# apt-get install nano
Reading package lists... Done
Building dependency tree
Reading state information... Done
Suggested packages:
  spell
The following NEW packages will be installed:
  nano
0 upgraded, 1 newly installed, 0 to remove and 20 not upgraded.
Need to get 194 kB of archives.
After this operation, 614 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main nano amd64 2.2.6-1ubuntu1 [194 kB]
Fetched 194 kB in 1s (173 kB/s)
Selecting previously unselected package nano.
(Reading database ... 37832 files and directories currently installed.)
Preparing to unpack .../nano_2.2.6-1ubuntu1_amd64.deb ...
Unpacking nano (2.2.6-1ubuntu1) ...
Setting up nano (2.2.6-1ubuntu1) ...
update-alternatives: using /bin/nano to provide /usr/bin/editor (editor) in auto mode
update-alternatives: using /bin/nano to provide /usr/bin/pico (pico) in auto mode
root@5eefb4b4c473:/# nano /conf/valhalla.json

Modified the config to include this:

    "tile_dir": "/data/valhalla",
    "tile_url": "http://com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com/north_america_2018_01_04/",
    "timezone": "/data/valhalla/tz_world.sqlite",

Now to start the server

root@5eefb4b4c473:/# /scripts/start_valhalla.sh
2018-01-06 12:34:37,575 CRIT Supervisor running as root (no user in config file)
2018/01/06 12:34:38.589690 [ERROR] bad upstream endpoint
2018/01/06 12:34:38.589741 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/06 12:34:39.602464 [ERROR] bad upstream endpoint
2018/01/06 12:34:39.602603 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/06 12:34:41.613965 [ERROR] bad upstream endpoint
2018/01/06 12:34:41.614021 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument
2018/01/06 12:34:44.627869 [ERROR] bad upstream endpoint
2018/01/06 12:34:44.627950 [ERROR] bad downstream endpoint
terminate called after throwing an instance of 'std::runtime_error'
  what():  Invalid argument

Those are the same errors as when it worked earlier.

Now try a request:

jesse:valhalla-docker/ (master) $ curl 'http://localhost:8002/route?json=\{%22locations%22:\[\{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22\},\{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22\}\],%22costing%22:%22auto%22\}'
{"error_code":171,"error":"No suitable edges near location","status_code":400,"status":"Bad Request"}%

it failed. The server logs dont include anything useful.

5 2018/01/06 12:40:27.571019 GET /route?json={%22locations%22:[{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22},{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22}],%22costing%22:%22auto%22} HTTP/1.1
2018/01/06 12:40:27.571211 [INFO] Got Loki Request 5
2018/01/06 12:40:27.571284 [ANALYTICS] locations_count::2
2018/01/06 12:40:27.571296 [ANALYTICS] costing_type::auto
2018/01/06 12:40:27.571361 [ANALYTICS] location_distance::22.380037km
2018/01/06 12:40:27.573849 [ANALYTICS] 400::No suitable edges near location
5 2018/01/06 12:40:27.574007 400 228

Maybe i got the format of the tile url wrong? Stop the server, and try again. I'll take another look in the source for the format of the url.

It looks like the method responsible for loading the tiles is in valhalla/src/baldr/graphtile.cc

GraphTile::GraphTile(const std::string& tile_url, const GraphId& graphid, curler_t& curler) {
  // Don't bother with invalid ids
  if (!graphid.Is_Valid() || graphid.level() > TileHierarchy::get_max_level())
    return;

  // Get the response returned from curl
  std::string uri = tile_url + "/" + FileSuffix(graphid.Tile_Base());
  long http_code;
  auto tile_data = curler(uri, http_code);

  // If its good try to use it
  if(http_code == 200) {
    graphtile_ = std::make_shared<std::vector<char> >(std::move(tile_data));
    Initialize(graphid, &(*graphtile_)[0], graphtile_->size());
    //TODO: optionally write the tile to disk?
  }
}

std::string uri = tile_url + "/" + FileSuffix(graphid.Tile_Base()); That looks like the problem, my url had a trailing /, and this is adding another one.

Start the server, try another request, and it still doesn't work.

It would be great if there was a way to log requested tile urls, but it doesnt seem like there is an easy way to do that. Maybe modify source code, do a local build, push to a fork on github, modify dockerfile-source to point at fork and branch, do do a build. Build fails, darn. Try something else.

Maybe the problem is having tile_dir and tile_url in the config. Tried removing tile_dir, now requests just timeout.

Tried running wireshark, it's not requesting any tiles.

Tried building locally and running, not in docker, from branch with curl verbose logging, and tiles get requested, but routing still fails.

server logs:

1 2018/01/08 13:37:31.923296 GET /route?json={%22locations%22:[{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22},{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22}],%22costing%22:%22auto%22} HTTP/1.1
2018/01/08 13:37:31.923563 [INFO] Got Loki Request 1
2018/01/08 13:37:31.923630 [ANALYTICS] locations_count::2
2018/01/08 13:37:31.923636 [ANALYTICS] costing_type::auto
2018/01/08 13:37:31.923677 [ANALYTICS] location_distance::22.380037km
*   Trying 52.216.0.90...
* TCP_NODELAY set
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/2/000/737/509.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: MP9A1EAVHRfu+JaV57X57uGvprxFUZxK23izWGVFXyHBTiG8yakenDI61Jkr6vPQZq6lF4qZShA=
< x-amz-request-id: A7BAD0DCC1B16019
< Date: Mon, 08 Jan 2018 13:37:33 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515132194/atime:1515179730/md5:8bac3e23cb328fa97de1940b4069ad5a/ctime:1515132194
< Last-Modified: Sat, 06 Jan 2018 12:45:52 GMT
< ETag: "8bac3e23cb328fa97de1940b4069ad5a"
< Content-Type: application/vnd.flographit
< Content-Length: 2626000
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/1/046/137.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: JOcPJcvzyJ5sDerLvoZOQPxmx+OOjFY4XGa+FGnQ6cJ4+N0KhGCsfx/mBOKLlItmqjRyqlyD1ng=
< x-amz-request-id: 74E7CAFCD17DF342
< Date: Mon, 08 Jan 2018 13:37:34 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515131129/atime:1515134022/md5:4162fa54abc449599723199beddb6ded/ctime:1515131129
< Last-Modified: Fri, 05 Jan 2018 19:38:00 GMT
< ETag: "4162fa54abc449599723199beddb6ded"
< Content-Type: application/vnd.flographit
< Content-Length: 4230384
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/0/002/894.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: Othk7T5ASzkh89vVeL3g2n9O2s2HRXohHXhhlZA9ChXyu7z41P3hPlfxDo/fbKPu2nV6PwjQZCo=
< x-amz-request-id: 4E320DFE9C20D629
< Date: Mon, 08 Jan 2018 13:37:36 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515131610/atime:1515134082/md5:bf31e633edfe6c1eb51052c2001dfb27/ctime:1515131610
< Last-Modified: Fri, 05 Jan 2018 19:21:38 GMT
< ETag: "bf31e633edfe6c1eb51052c2001dfb27"
< Content-Type: application/vnd.flographit
< Content-Length: 5895896
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/1/045/777.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: pEBF2TAlxDieoQGunaYEbVpRNjbcBDU7yL9q8u6Qor+VVpZ6DBHA1FYi5iUfBFsGPVpn2AIT3K0=
< x-amz-request-id: 1B2B62EEFE6B2E0F
< Date: Mon, 08 Jan 2018 13:37:38 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515131947/atime:1515134027/md5:f460de4fff4a925601b956ae8ccfe96e/ctime:1515131947
< Last-Modified: Fri, 05 Jan 2018 19:36:16 GMT
< ETag: "f460de4fff4a925601b956ae8ccfe96e"
< Content-Type: application/vnd.flographit
< Content-Length: 11301520
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/2/000/736/069.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: IdvwndOW3nF4unLcRcmhZ451DFa1Ab8s5rPZimDOH1Zj/VzQm9UJVWF8TtT7DWtICtTfwMS7P54=
< x-amz-request-id: ACF654B0F704F047
< Date: Mon, 08 Jan 2018 13:37:45 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515132194/atime:1515179729/md5:b9843fe562fa2ff7944d12f2e0a138ad/ctime:1515132194
< Last-Modified: Sat, 06 Jan 2018 12:44:40 GMT
< ETag: "b9843fe562fa2ff7944d12f2e0a138ad"
< Content-Type: application/vnd.flographit
< Content-Length: 2589872
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/2/000/737/510.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: 4pZWDCVNHPRVfI0QvffH36XUfAjN0FnD6f1weyfkzfvtDobwuY5elwWXarpB2Q7l3eNo1abwLb8=
< x-amz-request-id: B7171472598EE581
< Date: Mon, 08 Jan 2018 13:37:46 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515132194/atime:1515179730/md5:9be80051bf58a7f5e39931c104a60c91/ctime:1515132194
< Last-Modified: Sat, 06 Jan 2018 12:45:54 GMT
< ETag: "9be80051bf58a7f5e39931c104a60c91"
< Content-Type: application/vnd.flographit
< Content-Length: 1457016
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/0/002/804.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: Uf0Q0OgirTk8sTAmuJK3gIw510ixixptcLuUjifXbnEDusT3iybRQTrWXRfmrlCB72yFDch2lx4=
< x-amz-request-id: 0018CD759A20B7C9
< Date: Mon, 08 Jan 2018 13:37:47 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515132012/atime:1515134092/md5:201331c46f82f234a11d4a169fc507db/ctime:1515132012
< Last-Modified: Fri, 05 Jan 2018 19:20:01 GMT
< ETag: "201331c46f82f234a11d4a169fc507db"
< Content-Type: application/vnd.flographit
< Content-Length: 10573760
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
* Found bundle for host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com: 0x7fbbff05c370 [can pipeline]
* Re-using existing connection! (#0) with host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
* Connected to com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com (52.216.0.90) port 80 (#0)
> GET /north_america_2018_01_04/2/000/738/949.gph HTTP/1.1
Host: com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com
Accept: */*
Accept-Encoding: deflate, gzip

< HTTP/1.1 200 OK
< x-amz-id-2: AWygElY56brezcm+bvuwlps3EDx57JiIhY8+tmp8PqUAGojEsmbNyEYSZnMjXBCM69AvFDVtyEA=
< x-amz-request-id: 6F88C435D2186EAA
< Date: Mon, 08 Jan 2018 13:37:53 GMT
< x-amz-meta-s3cmd-attrs: uid:501/gname:staff/uname:jesse/gid:20/mode:33188/mtime:1515132194/atime:1515179730/md5:9ab44c48586797358a40d9e435b8d558/ctime:1515132194
< Last-Modified: Sat, 06 Jan 2018 12:47:25 GMT
< ETag: "9ab44c48586797358a40d9e435b8d558"
< Content-Type: application/vnd.flographit
< Content-Length: 4511680
< Server: AmazonS3
<
* Connection #0 to host com.gaiagps.routingdata.s3-website-us-east-1.amazonaws.com left intact
2018/01/08 13:37:54.603726 [ANALYTICS] 400::Locations are in unconnected regions. Go check/edit the map at osm.org
1 2018/01/08 13:37:54.603939 400 268

response:

{"error_code":170,"error":"Locations are in unconnected regions. Go check\/edit the map at osm.org","status_code":400,"status":"Bad Request"}

At least thats a new error, and tiles are getting fetched. Maybe a settings issue, or an issue parsing tiles? Setting loki.use_connectivity = false in the config file fixed it.

So it works if i build from source, but doing a docker build from source doesnt work. I opened a github issue, https://github.com/valhalla/docker/issues/28.

Tried again in docker, this time extending the valhalla/docker:ppa-1.4.4 image, and adding a config file. Still doesnt work. No tiles are getting requested.

Maybe there is something in the recent commit history. Sure enough, tile_url was added since the last release. So thats why a build from source works, but the last release doesn't.

Check github notifications, there's a comment on the build failure issue, and one of the valhalla devs(Kevin Kreiser) commited a fix. https://github.com/valhalla/valhalla/commit/8758766269132c88ba883b5507a728618fddeaa2 Fixed within 15 minutes of opening the issue!

But it still doesnt build in docker. Now there's another similar issue. Commented on issue again.

Fixed the issue https://github.com/valhalla/valhalla/pull/1066 PR is merged, and now it's working!

Now im getting somewhere.

Next goal: Get it running on AWS

Made a new private github repo with a Dockerfile and a config file for valhalla. Later this will autobuild on codeship, but for now i'll build it manually.

new dockerfile:

FROM valhalla-source
RUN mkdir /opt/data/
ADD tz_world.sqlite /opt/data
ADD valhalla.json /conf

The valhalla-source that it's refering to is a local build(docker build -t valhalla-source -f Dockerfile-source), but once there is a new release it will be FROM valhalla/docker:source.

Made a new docker repo on AWS ecr, lookup the push command on the aws console, then build, tag, and push

jesse:valhalla-docker-gaia/ $ `aws ecr get-login --no-include-email --region us-east-1`

jesse:valhalla-docker-gaia/ $ docker build -t gaiagps/valhalla .
Sending build context to Docker daemon  40.73MB
Step 1/4 : FROM valhalla-source
 ---> 9559c611af3c
Step 2/4 : RUN mkdir /opt/data/
 ---> Running in d9f2a72bc0a1
 ---> 6419e260ca39
Removing intermediate container d9f2a72bc0a1
Step 3/4 : ADD tz_world.sqlite /opt/data
 ---> 2cf67e0afa75
Step 4/4 : ADD valhalla.json /conf
 ---> ba1714327522
Successfully built ba1714327522
Successfully tagged gaiagps/valhalla:latest

jesse:valhalla-docker-gaia/ $ docker tag gaiagps/valhalla:latest 265147712162.dkr.ecr.us-east-1.amazonaws.com/gaiagps/valhalla:latest
jesse:valhalla-docker-gaia/ $ docker push 265147712162.dkr.ecr.us-east-1.amazonaws.com/gaiagps/valhalla:latest
The push refers to a repository [265147712162.dkr.ecr.us-east-1.amazonaws.com/gaiagps/valhalla]
cbc4e805e107: Preparing
1f25ba8ab7cc: Pushed
86b1cdca2c0f: Pushed
052e2046012b: Pushed
dc098bfa5ece: Pushed
c67cd22b23de: Pushed
e5287a1bba4d: Pushed
e152a5edf70c: Pushed
8bc061a3a971: Pushed
a3d9471df149: Pushed
4455b4d81934: Pushed
4622ee8f36ae: Pushed
1ae9c3d1e0b7: Pushed
150e95c79e16: Pushed
1b0c71361973: Pushed
latest: digest: sha256:687f70eb6ebd159ee19f266af50e651a303991d78cc012d7af6fae4d683d0ed3 size: 3454

Made a new AWS ALB load balancer, with a https listener, and a single target group going to port 8222 over http, in an existing VPC.

Made a new task definition named routing-production

{
  "executionRoleArn": null,
  "containerDefinitions": [
    {
      "dnsSearchDomains": null,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/routing-production",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "entryPoint": null,
      "portMappings": [
        {
          "hostPort": 8222,
          "protocol": "tcp",
          "containerPort": 8002
        }
      ],
      "command": null,
      "linuxParameters": null,
      "cpu": 0,
      "environment": [],
      "ulimits": null,
      "dnsServers": null,
      "mountPoints": [],
      "workingDirectory": null,
      "dockerSecurityOptions": null,
      "memory": null,
      "memoryReservation": 1024,
      "volumesFrom": [],
      "image": "265147712162.dkr.ecr.us-east-1.amazonaws.com/gaiagps/valhalla:latest",
      "disableNetworking": null,
      "essential": true,
      "links": null,
      "hostname": null,
      "extraHosts": null,
      "user": null,
      "readonlyRootFilesystem": null,
      "dockerLabels": null,
      "privileged": null,
      "name": "valhalla"
    }
  ],
  "placementConstraints": [],
  "memory": null,
  "taskRoleArn": null,
  "compatibilities": [
    "EC2"
  ],
  "taskDefinitionArn": "arn:aws:ecs:us-east-1:265147712162:task-definition/routing-production:2",
  "family": "routing-production",
  "requiresAttributes": [
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.ecr-auth"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.awslogs"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.21"
    },
    {
      "targetId": null,
      "targetType": null,
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19"
    }
  ],
  "requiresCompatibilities": [
    "EC2"
  ],
  "networkMode": "bridge",
  "cpu": null,
  "revision": 2,
  "status": "ACTIVE",
  "volumes": []
}

Made a new service on an existing ECS cluster, using the routing-production task definition, and registered with the new load balancer. Press save, wait a couple minutes.

Check the logs, and it looks like the service started up.

jesse:valhalla-docker-gaia/ (master✗) $ awslogs get /ecs/routing-production
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018-01-08 17:32:16,631 CRIT Supervisor running as root (no user in config file)
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:18.003828 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:18.003904 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:18.249739 [WARN] Tile extract could not be loaded
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:18.254287 [WARN] Tile extract could not be loaded
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:19.267327 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:19.267741 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:21.297927 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:21.298600 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:23.325362 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:23.338673 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:24.373609 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:24.374084 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:26.394527 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:26.395232 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:29.417611 [ERROR] bad upstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 17:32:29.418169 [ERROR] bad downstream endpoint
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 terminate called after throwing an instance of 'std::runtime_error'
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306   what():  Invalid argument

Make a request, and it works!

jesse:valhalla/ (master) $ curl 'https://routing.gaiagps.com/route?json=\{%22locations%22:\[\{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22\},\{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22\}\],%22costing%22:%22auto%22\}' -v > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 52.86.249.167...
* TCP_NODELAY set
* Connected to routing.gaiagps.com (52.86.249.167) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.gaiagps.com
* Server certificate: Amazon
* Server certificate: Amazon Root CA 1
* Server certificate: Starfield Services Root Certificate Authority - G2
> GET /route?json={%22locations%22:[{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22},{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22}],%22costing%22:%22auto%22} HTTP/1.1
> Host: routing.gaiagps.com
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Mon, 08 Jan 2018 20:44:02 GMT
< Content-Type: application/json;charset=utf-8
< Content-Length: 7094
< Connection: keep-alive
< Access-Control-Allow-Origin: *
<
{ [4225 bytes data]
100  7094  100  7094    0     0  13636      0 --:--:-- --:--:-- --:--:-- 13642
* Connection #0 to host routing.gaiagps.com left intact

check the server logs, and it's in there.

/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 1200 2018/01/08 20:44:02.762340 GET /route?json={%22locations%22:[{%22lat%22:38.19987817399173,%22lon%22:-122.59918212890626,%22type%22:%22break%22},{%22lat%22:38.00427891593763,%22lon%22:-122.54013061523438,%22type%22:%22break%22}],%22costing%22:%22auto%22} HTTP/1.1
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.764722 [INFO] Got Thor Request 1200
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.762794 [INFO] Got Loki Request 1200
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.762882 [ANALYTICS] locations_count::2
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.762891 [ANALYTICS] costing_type::auto
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.762966 [ANALYTICS] location_distance::22.380037km
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.765442 [ANALYTICS] travel_mode::0
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.813787 [ANALYTICS] admin_state_iso::
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.814317 [ANALYTICS] admin_country_iso::
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.814961 [INFO] Got Odin Request 1200
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.815343 [INFO] trip_path_->node_size()=39
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 2018/01/08 20:44:02.816251 [INFO] maneuver_count::12
/ecs/routing-production ecs/valhalla/8d193ae0-2134-4e51-bc05-053fe0b8a306 1200 2018/01/08 20:44:02.817176 200 7213

Next goal: Make it faster

It looks like the tiles can be gziped, and there is significant savings from doing that, so i'll write a script to do a parallel upload, gzip the tiles, and set the appropriate headers on s3.

It seems to be fetching routing tiles for every request and not doing any caching, that could get expensive, need to look into that.

Adapted an existing script for uploading tiles from an MBTiles file to upload routing tiles instead. Uploads in parallel with multiple threads, gzip compresses before uploading, and sets the correct headers. Script lives in private github repo, but it's also posted to https://gist.github.com/JesseCrocker/4afd4769197d714af3e4788d948401ac

Uploaded the north america tileset again, this timne with the gziping script, modified config file to point to it, did a docker build, ran locally, tested, and it's working.

In testing if the gzipped tiles were working I made multiple requests in a row, and it seemed like subsequent requests were fast, so maybe tiles are being cached. Start up wireshark, start capturing on en0, apply a display filter of http.request.method == "GET", make a couple requests, and it looks like tile caching is working. The fist request fetches a bunch of tiles, subsequent requests aren't requesting any tiles? Maybe it's caching the route response and not the tiles? Try changing costing of my sample request from auto to pedestrian, 1 more tile is requested, not the 6 tiles from the original request. Then change costing to bicycle, and no new tiles are requested. So i guess caching is working.

Now i'll commit the new config file, do a new docker build, push it, and update the task definition and service on AWS. Updated, it works, seems to be a little faster.

Next goal: Get tile slicing running on aws

First step: copy scripts/cut_tiles.sh into new github repo, and modify it to remove steps that i wont be using, like transit, and rebuilding timezones everytime, make it download the extract, and use new upload script.

Next goal: Autoscaling

Next goal: Automatic builds running on codeship

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment