Skip to content

Instantly share code, notes, and snippets.

Avatar

Jesse Crocker JesseCrocker

View GitHub Profile
@JesseCrocker
JesseCrocker / check_urls.py
Created Jan 31, 2017
check a list of urls to see if they exist
View check_urls.py
#!/usr/bin/env python
import logging
from optparse import OptionParser
import os
import requests
import sys
def check_urls_from_file(f):
statuses = {}
@JesseCrocker
JesseCrocker / gist:46436874baef539e587c7033bea1f5c5
Last active Sep 22, 2016
Shell script for merging a bunch of shapefiles into 1
View gist:46436874baef539e587c7033bea1f5c5
file=merge.shp
SRS=epsg:4326
for i in boundary/**/*.shp
do
if [ ! -e "$file" ]
then
echo "creating $file"
ogr2ogr -t_srs $SRS -f 'ESRI Shapefile' $file $i
else
echo "merging $i into $file"
@JesseCrocker
JesseCrocker / tilenames.py
Last active Aug 25, 2017
Translates between lat/long and the slippy-map tile numbering scheme - tilenames.py -t 16/10470/25324 -g | geojsonio
View tilenames.py
#!/usr/bin/env python
#-------------------------------------------------------
# Translates between lat/long and the slippy-map tile
# numbering scheme
#
# http://wiki.openstreetmap.org/index.php/Slippy_map_tilenames
#
# Written by Oliver White, 2007
# This file is public-domain
#-------------------------------------------------------
@JesseCrocker
JesseCrocker / localconfig.js
Created Aug 16, 2016
local config file for changing source of gaia topo
View localconfig.js
exports.LocalConfig = function (localizer, project) {
var mode = process.env.SOURCE_OVERRIDE;
if(mode == "local_composite") {
project.mml.source[0] = {
protocol: "http:",
tilejson: "http://localhost:8080/meters/index.json"
}
} else if(mode == "local_source") {
project.mml.source[0] = {
protocol: "tmsource:",
@JesseCrocker
JesseCrocker / findings.md
Last active Jul 29, 2016
Benchmarking tile server performance
View findings.md

Purpose

Benchmarking rendering vector tiles into rasters, because tile requests are currently taking way to long. CDN reports the average latency for backend requests is around 1 second.

Setup

A single AWS ec2 m4.large instance running tessera tile server. Tile server is running in the foreground, in screen. Instance is placed in a security group with port 8080 exposed to the internet.

Requests are being made over the Internet, from my home computer.

Benchmarking is being done with a lightly modified version of palb, The modification logs urls that return an error.

@JesseCrocker
JesseCrocker / add-halo.py
Last active May 27, 2016
Add a white halo around maki icons
View add-halo.py
#!/usr/bin/env python
import logging
from optparse import OptionParser
import os
from xml.dom.minidom import parse
import xml.dom.minidom
import copy
def add_halo(in_file, out_file, width, stroke_color):
@JesseCrocker
JesseCrocker / convert-image-urls-to-data.py
Created May 10, 2016
Download an html page from a url and write out a copy with all linked images converted to data:// urls
View convert-image-urls-to-data.py
#!/usr/bin/env python
import os
import re
from bs4 import BeautifulSoup
import urllib2
from urlparse import urljoin
from optparse import OptionParser
import logging
def url_can_be_converted_to_data(tag):
View ClipperTest.h
//
// ClipperTest.h
// TrailTracker
//
// Created by Jesse Crocker on 5/2/16.
//
//
#import <Foundation/Foundation.h>
@JesseCrocker
JesseCrocker / function.js
Created Apr 29, 2016
AWS lambda function for posting cloudwatch alarms to slack
View function.js
var AWS = require('aws-sdk');
var url = require('url');
var https = require('https');
var hookUrl, kmsEncyptedHookUrl, slackChannel;
kmsEncyptedHookUrl = ''; // Enter the base-64 encoded, encrypted key (CiphertextBlob)
slackChannel = '#alerts'; // Enter the Slack channel to send a message to
@JesseCrocker
JesseCrocker / publish_queue_size.py
Created Apr 8, 2016
Publish rabbitMQ queue depths as AWS cloudwatch metrics
View publish_queue_size.py
#!/usr/bin/env python
from __future__ import with_statement, print_function
from pyrabbit.api import Client
from boto.ec2.cloudwatch import CloudWatchConnection
import os
from time import sleep
def get_queue_depths(host, username, password, vhost):
cl = Client(host, username, password)