Skip to content

Instantly share code, notes, and snippets.

View mfurlend's full-sized avatar

Mikhail Furlender mfurlend

  • WeatherBELL Analytics
  • New York, NY
View GitHub Profile
@mfurlend
mfurlend / README.md
Created December 9, 2021 18:29
Use native mysql client library in AWS Lambda through serverless

Non-native mysql clients such as pymysql perform much slower than the native mysql client. Here’s how to get the native client working with lambda (through serverless):

  1. Download layers.zip from this repo https://github.com/nonbeing/mysqlclient-python3-aws-lambda/tree/master/build_output
  2. Move it to <project_root>/layers/mysqlclient.zip
  3. Add this to serverless.yml:
layers:
 mysqlclient:
   name: list-lambda-${opt:stage}-mysqlclient
package:
@mfurlend
mfurlend / csv_to_shp.sh
Created September 19, 2019 19:47
CSV to SHP
ogr2ogr -s_srs EPSG:4326 -t_srs EPSG:3857 -oo X_POSSIBLE_NAMES='LON*' -oo Y_POSSIBLE_NAMES='LAT*' -f "ESRI Shapefile" out.shp ghcn.csv
@mfurlend
mfurlend / georeference_and_crop_example.sh
Created May 29, 2019 21:28
georeference and crop an image
gdal_translate -of GTiff -a_ullr ullon ullat lrlon lrlat -a_srs EPSG:4269 input.tif output.tif
gdal_translate -of GTiff -a_ullr -134.09548 52.640547 -60.90728 21.14054 -a_srs EPSG:102009 ~/Desktop/example_image.png output.tif
gdalwarp -t_srs EPSG:4326 -te ullon ullat lrlon lrlat -ts width_px height_px input.tif output.tif
gdalwarp -t_srs EPSG:102009 -te -100 30 -80 40 -ts 920 768 output.tif output2.tif
@mfurlend
mfurlend / gcp.sh
Created January 16, 2019 14:43
Fast google cloud VM ssh sessions
gcp () {
local inp1 inp2
inp1=$1
inp2=$2
DEFAULT_USER=SOME_USER
if (( ! $# ))
then
inp1="-h"
fi
if [[ $inp1 == "-h" || $inp1 == "--help" ]]
@mfurlend
mfurlend / goo.sh
Created January 16, 2019 14:41
Fast google cloud profile switching/listing
goo () {
local inp1 inp2
inp1=$1
inp2=$2
if (( ! $# ))
then
inp1="-h"
fi
if [[ $inp1 == "-h" || $inp1 == "--help" ]]
then
@mfurlend
mfurlend / eachhost.sh
Last active January 7, 2019 15:42
run command on each host
# run command on each host in /etc/hosts and print the output
# stop on failure (remove || break to continue on failure)
# use a pretty line break to delimit responses
# like this:
#┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┫hostname┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
eachhost () {
# ignore line 1 and 5 in /etc/hosts
for host in $(sed 1,5d /etc/hosts|awk '{print $2}')
do
(
@mfurlend
mfurlend / webtorrent-desktop-batch-delete.js
Created October 4, 2018 16:19
webtorrent-desktop batch delete scripts
//Delete all
let delete_buttons = Array.from(document.getElementsByClassName('delete'))
while (delete_buttons.length > 0) {
let button = delete_buttons.pop();
button.click();
Array.from(document.querySelector('.control.ok').getElementsByTagName('*')).forEach(o=>o.click());
}
// Delete paused
@mfurlend
mfurlend / nfs_shares_with_subdirectories.md
Last active June 8, 2018 17:16
Export/Import NFS shares with subdirectories as mount points
/etc/exports (on NFS server)
/mnt/disk                     10.142.0.0/24(rw,no_subtree_check,sync,crossmnt)
/mnt/disk/WRFV381/data_ingest 10.142.0.0/24(rw,no_subtree_check,sync)
/etc/auto.hvm (on NFS client, with AutoFS)
/mnt/disk                     -rw,soft,intr,rsize=8192,wsize=8192,nosuid    10.142.0.12:/mnt/disk
/mnt/disk/WRFV381/data_ingest -rw,soft,intr,rsize=8192,wsize=8192,nosuid    10.142.0.12:/mnt/disk/WRFV381/data_ingest
@mfurlend
mfurlend / delete_all_torrents_in_webtorrent.js
Created May 14, 2018 18:04
delete all torrents in WebTorrent
var delete_buttons = Array.from(document.getElementsByClassName('delete'))
while (delete_buttons.length > 0) {
var button = delete_buttons.pop();
button.click();
Array.from(document.querySelector('.control.ok').getElementsByTagName('*')).forEach(o=>o.click());
}
@mfurlend
mfurlend / save-to-datastore-index.js
Created February 6, 2018 03:13
#google #datastore #save #cloud functions
exports.transaction = function transaction (req, res) {
// Imports the Google Cloud client library
const Datastore = require('@google-cloud/datastore');
// Your Google Cloud Platform project ID
const projectId = 'moneypenny-dabc6';
// Instantiates a client
const datastore = Datastore({
projectId: projectId