View README.md

Deploy updates to aws ecs services based on github commits. Posts deploy notifications to slack.

Assumptions:

  • Task definitions map 1:1 with services, and they have the same name.
  • Code is stored on github
  • You want to deploy the latest commit in a branch
  • Docker images are tagged with commit SHA
  • Docker images are stored on AWS ECR
View upload_tiles.py
#!/usr/bin/env python3
import os
from urllib.parse import urlparse
from multiprocessing.pool import ThreadPool
from functools import partial
import gzip
from boto.s3.connection import S3Connection
from boto.s3.key import Key
View valhalla diary.md

Ultimate Goal: valhalla running on AWS ECS, behind a load balancer, in multiple regions, with auto scaling, with data updating once a week.

Diary

First goal: Build with docker, then cut tiles from a small extract and get server to run locally.

jesse:projects/ $ mkdir ~/valhalla-data
jesse:projects/ $ cd ~/valhalla-data
jesse:projects/ $ wget http://download.geofabrik.de/north-america/us/california-latest.osm.pbf
[output abbreviated]
jesse:projects/ $ cd ~/projects
View split.py
#!/usr/bin/env python
import logging
from optparse import OptionParser
import os
import gpxpy
import gpxpy.gpx
def split_gpx(source, dest_dir, max_segment_points=500):
View ogr_value_summary.py
#!/usr/bin/env python
import logging
from optparse import OptionParser
import os
from osgeo import ogr
def log_file_fields(filename):
print("File: " + filename)
source = ogr.Open(filename)
View load.py
#!/usr/bin/env python3
import os
import psycopg2
import re
import sys
from urllib.parse import urlparse
try:
conn = psycopg2.connect(dbname=os.environ.get("DB"), user=os.environ.get("DB_USER"), host=os.environ.get("DB_HOST"))
View cleanup_edges.py
#!/usr/bin/env python
import numpy
import sys
import gdal
def cleanup_edges(source, nodata_value, mask_filename="mask.tiff"):
band_data = source.GetRasterBand(1).ReadAsArray()
y_size, x_size = band_data.shape
View profile-source.py
#!/usr/bin/env python
import logging
from optparse import OptionParser
import os
import sys
import mercantile
import psycopg2
import yaml
View test.py
#!/usr/bin/env python
import subprocess
def _main():
source_image = "source.tiff"
subprocess.call("gdalwarp -t_srs epsg:3857 -r cubic " + source_image + " pre-import.tiff", shell=True)
import fiona
subprocess.call("gdalwarp -t_srs epsg:3857 -r cubic " + source_image + " post-import.tiff", shell=True)
View StyleTemplate.json
{
"version": 8,
"name": "OpenHuntingData",
"center": [
-118.291995,
36.578581
],
"zoom": 12,
"bearing": 0,
"pitch": 0,