Skip to content

Instantly share code, notes, and snippets.

View shaystrong's full-sized avatar

Shay Strong shaystrong

  • Helsinki, Finland
  • 07:24 (UTC +03:00)
View GitHub Profile
@shaystrong
shaystrong / silly_lib.py
Created March 26, 2020 18:15
silly madlibs with NLTK tokenizer
from nltk.tokenize import sent_tokenize, word_tokenize
from nltk.tokenize.treebank import TreebankWordDetokenizer
import argparse
#python sillylib.py --input bed button kill be black open clothes also
# inspired by https://towardsdatascience.com/a-i-plays-mad-libs-and-the-results-are-terrifying-78fa44e7f04e
parser = argparse.ArgumentParser()
parser.add_argument('--input',nargs='+',default=['bed','button','kill','be','black','open','clothes','even'])
parser.add_argument('--person',default='AI')
@shaystrong
shaystrong / cleanup
Created May 8, 2019 05:20
cleanup images & xml label pairs for object detectors
def cleanupPairs(pathh):
import glob
for i in glob.glob(pathh+'JPEGImages/'+'*png'):
filename_split = os.path.splitext(i)
filename_zero, fileext = filename_split
basename = os.path.basename(filename_zero)
if not os.path.isfile(pathh+'Annotations/'+basename+'.xml'):
os.system('rm '+i)
for i in glob.glob(pathh+'Annotations/'+'*xml'):
filename_split = os.path.splitext(i)
@shaystrong
shaystrong / tms_utils
Last active May 8, 2019 05:19
geospatial boxes to VOC xml boxes
def ll2subpix(lat,long):
import numpy as np
n = 2 ** zoom
xtile = n * ((long + 180) / 360)
ytile = n * (1 - (np.log(np.tan(lat*np.pi/180) + 1/(np.cos(lat*np.pi/180))) / np.pi)) / 2
fracy,integery=np.modf(ytile)
fracx,integerx=np.modf(xtile)
return fracy,integery,fracx,integerx
def tmsVOCxml(dirr,label):
@shaystrong
shaystrong / building boxes
Created May 8, 2019 05:13
building polygons to rectangles
def footprint2box(vector):
df = vector
df['geometry']=df.envelope #round to nearest (axis-aligned) rectangle
df['class']='building' #all buildings we will just call class=buildings
return df
df=footprint2box(buildings)
@shaystrong
shaystrong / flattenTMS
Created May 8, 2019 05:10
'flatten' TMS nested xyz to single directory structure
def flattenTMS(tilepath,zoom,voc):
"""flatten the TMS tile structure to put all images in one directory
e.g. tilepath='tiles', zoom='18'"""
import glob,os
for i in glob.glob(tilepath+'/'+zoom+'/*/*png'):
filename_split = os.path.splitext(i)
filename_zero, fileext = filename_split
basename = os.path.basename(filename_zero)
strn=i.split('/')[-3]+'_'+i.split('/')[-2]+'_'+i.split('/')[-1]
#write_xml_annotation(strn, boxCoords)
@shaystrong
shaystrong / GDAL translate + mbtiles
Created May 8, 2019 05:03
translate a tif to mbtiles and unpack them
gdal_translate -of MBTiles nzwani_dg_wv3_05032019_104001004A465900.tif nzwani_dg_wv3_05032019_104001004A465900.mbtiles
mb-util nzwani_dg_wv3_05032019_104001004A465900.mbtiles tiles/
@shaystrong
shaystrong / vsicurl & COG
Created May 8, 2019 05:00
use GDAL to grab make a COG vrt
gdalbuildvrt foo.vrt /vsicurl/http://opendata.digitalglobe.com/cyclone-kenneth/pre-event/2017-10-28/103001007206B400/3023101.tif
%matplotlib inline
import osmnx as ox
import matplotlib.pyplot as plt
place_name = "Nzwani Centre, Comores"
graph = ox.graph_from_place(place_name, which_result=2)
fig, ax = ox.plot_graph(graph)
area = ox.gdf_from_place(place_name)
buildings = ox.buildings_from_place(place_name)

The Taco GraphTM

ML Team Contest, 1

I have this lovely ‘Keep Pursuing’ backpack with the new EV logo (retail $295). I wanted to give it to the person in our team that has a proven track record of (1) collaborating with others, (2) being helpful, & (3) adding true technical value to our team. So maybe we start by looking at the tacos people on our team have recieved.

alt text. alt text

But wait!...

import os
import geopandas as gpd
import pandas as pd
import boto3

os.system('cat my_roi_4326.geojson | supermercado burn 18 | mercantile shapes | fio collect > my_roi_4326_outtiles.geojson')

tiles = gpd.read_file('my_roi_4326_outtiles.geojson')
tiles.id=tiles.id.map(lambda x: x.lstrip('(').rstrip(')'))