Skip to content

Instantly share code, notes, and snippets.

View n0531m's full-sized avatar
💭
in the ☁

Naoya Moritani n0531m

💭
in the ☁
3 followers · 25 following
  • Singapore
View GitHub Profile
@n0531m
n0531m / gist:9025a52c8bd4d87ebeeb1669d2822fa3
Last active November 28, 2017 05:18
Clean macports env
#https://superuser.com/questions/165652/how-can-i-clean-up-my-macports-installation
sudo /opt/local/bin/port -f clean --all all
sudo /opt/local/bin/port -f uninstall inactive
sudo rm -rf /opt/local/var/macports/distfiles/*
sudo rm -rf /opt/local/var/macports/packages/*
sudo /opt/local/bin/port uninstall inactive
@n0531m
n0531m / runBeamPipeline.sh
Created November 15, 2017 14:47
running a beam pipeline from cli
#!/bin/bash
PROJECTID=moritani-bigdata
DATASET=opendatasg
RUNNER=DirectRunner
#RUNNER=DataflowRunner
mvn compile exec:java \
-Dexec.mainClass=com.gmail.n0531m.datagovsg.pipelines.TaxiAvailabilityPipeline \
-Dexec.cleanupDaemonThreads=false \
@n0531m
n0531m / bigquery_udf_latlng2coordinates.sql
Last active October 30, 2017 00:46
calculate coordinates from latlng
# this is a udf sample to calculate coordinates from latlng
# https://developers.google.com/maps/documentation/javascript/examples/map-coordinates
CREATE TEMPORARY FUNCTION
project (latitude FLOAT64, longitude FLOAT64, zoom FLOAT64)
RETURNS STRUCT < worldCoordinate_x float64,
worldCoordinate_y float64,
pixelCoordinate_x float64,
pixelCoordinate_y float64,
tileCoordinate_x float64,
tileCoordinate_y float64 >
@n0531m
n0531m / datalab_update.sh
Created May 21, 2017 01:29
update local datalab container image
#!/bin/bash
docker pull gcr.io/cloud-datalab/datalab:local
@n0531m
n0531m / datalab_start.sh
Last active May 21, 2017 01:28
start cloud datalab container image locally
#!/bin/bash
docker run -it -p 127.0.0.1:8081:8080 -v $HOME/datalab:/content -e PROJECT_ID=sandbox-moritani gcr.io/cloud-datalab/datalab:local
@n0531m
n0531m / start_syntaxnet.sh
Created May 21, 2017 01:26
start a docker container with SyntaxNet
#!/bin/bash
docker run -it -p 8888:8888 tensorflow/syntaxnet
@n0531m
n0531m / copyDoc.js
Created December 14, 2016 08:33
AppsScript to copy doc if updated
//This script should be stored in Drive as a AppScript file and configured to run at intervals
//In my case, run it every two hours and copied the doc if there was an update
//ID of original document
var srcDocId="xxxxxx";
//ID of desitination Folder
var targetFolderId= "xxxxxx";
function copyDoc() {
copyDocIfUpdated(srcDocId,targetFolderId,2);
@n0531m
n0531m / gist:bf0437e06b1c1c697cd1b82a7cb8c39a
Created September 28, 2016 00:12
To see which processes (files) are consuming a range of ports, use the List Open Files
# To see which processes (files) are consuming a range of ports, use the List Open Files command
lsof -i :8000-8020
@n0531m
n0531m / BQ to SpreadSheet.js
Last active June 11, 2020 04:41
Google Apps Script that updates spreadsheet and shows chart based on BigQuery query
// replace with project number
var projectNumber = 'xxxxxxx';
// replace with Sheets's ID
var ssid = "xxxxxxx";
var ss = SpreadsheetApp.getActiveSpreadsheet();
function getSpreadsheetId(){
@n0531m
n0531m / list_gcp_iprange.sh
Last active April 17, 2024 21:50
Google Cloud Platform : ip address range
#!/bin/bash
# https://cloud.google.com/compute/docs/faq#find_ip_range
# nslookup -q=TXT _cloud-netblocks.googleusercontent.com 8.8.8.8
myarray=()
for LINE in `dig txt _cloud-netblocks.googleusercontent.com +short | tr " " "\n" | grep include | cut -f 2 -d :`
do
myarray+=($LINE)
for LINE2 in `dig txt $LINE +short | tr " " "\n" | grep include | cut -f 2 -d :`