akiatoji

In GCP Console, create VPC networks named vpn-1 and vpn-2 in us-east1 and europe-west1

In Cloud Shell:

Create VPN gateways

gcloud compute target-vpn-gateways \
create vpn-1 \

akiatoji

OSRM North America route server on EC2

Overview

OSRM route server is an extremely useful tool for getting the driving distance/time through multiple locations. The route server requires data that has to be downloaded and processsed before it can be used to serve routes from.

Processing OSRM data for large region like North America can be a real challenge due to memory and disk size requirements. It's also really time consuming. If you cut and try from scratch, you will repeatedly run into some constraints and fail after hours of running.

The following are summary notes from trying this with eventual success.

akiatoji

Clojure on Raspberry Pi with OS X

"Clojure running on Raspberry Pi" sounded so cool that I just had to give it a try.

Install JDK

• Download ARM JDK from Oracle and instlal on Raspberry Pi
• Change visudo to contain the following

akiatoji

Dev setup for Kafka and Spark Streaming

This note is for setting up rapid prototyping of MongoDB -> Kafka -> Spark Streaming pipeline.

Pyspark is being used for fast turn around. Pyspark uses Kafka connector that uses ZooKeeper so the topic has to be created with offsets stored in ZooKeeper.

Prerequisites

$> brew install zookeeper kafka apache-spark jupyter

akiatoji

GCP VPC tunnel

Notes from Qwiklabs: Optimizing networks on GCP

Creating VPC tunnels manually. This is not how you connect VPC's in GCP, but this shows what all the pieces are.

Create two VPCs with a subnet and firewall rules

gcloud compute networks create cloud —subnet-mode custom
gcloud compute firewall-rules create cloud-fw —network cloud —allow tcp:22,tcp:5001,udp:5001,icmp

akiatoji

This is how I set up GCP VM and my Deep Learning PC for ML and Geo work.

On GCP, I normally use a n1-highmem-4 (4xvCPU, 26GB) with Tesla T4 GPU ($0.4/hr)

My own Deep Learning PC is a Ryzen 3900 (12 cores) 64GB memory and RTX2070 Super ($3K)

Install conda

Install MiniConda.

akiatoji

Particle Boron Setup Notes

Notes for using Particle.io Boron IoT via USB.

Flashing OTA has been a major paint for me as Boron 2G/3G can take forever before connecting to network. During development, it is best to use USB instead of Cell connection to Flash code.

Install Particle CLI

brew install dfu-util

akiatoji

import org.mongodb.scala.bson.collection.mutable.Document
import org.mongodb.scala.{MongoClient, MongoCollection, MongoDatabase}
import org.mongodb.scala.model.Aggregates._
import org.mongodb.scala.model.Accumulators._
import com.profuzio.MongoHelper._

val mongoClient: MongoClient = MongoClient.apply("mongodb://localhost")
val database: MongoDatabase = mongoClient.getDatabase("kiwanda")
val collection: MongoCollection[Document] = database.getCollection("labor")

akiatoji

sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target
sudo systemctl set-default multi-user.target

akiatoji

Debug Horizontal Scroll

(function (d) {
    var w = d.documentElement.offsetWidth,
        t = d.createTreeWalker(d.body, NodeFilter.SHOW_ELEMENT),
        b;
    while (t.nextNode()) {
        b = t.currentNode.getBoundingClientRect();
 if (b.right > w || b.left < 0) {