akiatoji

Install conda

Install MiniConda.

Install dependencies

These are the go-to packages I use for Machine Learning/Geo/NLTK work.

conda config --add channels conda-forge   # look in .condarc and make sure this is the first one 

akiatoji

Notes on how to use AWS SNS:

1.  Subscribe an HTTP endpoint (i.e. http://myhost/sns_endpoint) on AWS Console
2.  AWS will send subscription confirmation right away
3.  SNS_controller responds to subscription confirmation by sending confirmation using Fog.  
4.  Once AWS is happy, you can start sending notifications to your end point via SNS. 

akiatoji

# The Dockerfile used in https://coral.ai/docs/edgetpu/retrain-classification/ is a bit old and does not work

# Below is the updated Dockerfile to retrain image classifier for use in Coral EdgeTPU

FROM tensorflow/tensorflow:1.11.0-devel-py3

# Install wget (to make life easier below) and editors (to allow people to edit
# the files inside the container)
RUN apt-get update && \
    apt-get install -y wget vim emacs nano git && \

akiatoji

# Sometimes EarlyStopping is too much hassle

class myCallback(tf.keras.callbacks.Callback):
    def on_epoch_end(self, epoch, logs={}):
        if (logs.get('acc') >= 0.998):
            print("\nReached 99.8% accuracy so cancelling training!")
            self.model.stop_training = True
cb = myCallback()

akiatoji

In Cloud Console:

• Create load balancer
• Create Compute instances with network tag to use as load balancer target.
• Create health check

Create target pool

gcloud compute target-pools create extloadbalancer  --region $MY_REGION --http-health-check webserver-health

akiatoji

ENV APP_HOME /app ENV PORT 8080 CMD exec gunicorn --bind 0.0.0.0:$PORT \
                         --workers 1 --chdir $APP_HOME --threads 1 app:app

akiatoji

#!/usr/bin/env bash
gcloud services enable cloudkms.googleapis.com

KEYRING=keyring
KEY=secrets

gcloud kms keys list --location global --keyring $KEYRING

if [[ $? -ne 0 ]]; then
    gcloud kms keyrings create ${KEYRING} --location global

akiatoji

def load_from_encrypted(app):
    tempname = str(uuid.uuid4())
    encrypted_file_name = "secrets-%s.cfg.encrypted" % tempname
    decrypted_file_name = "secrets-%s.cfg.decrypted" % tempname

    storage_client = storage.Client()
    bucket = storage_client.get_bucket('vault')
    blob = bucket.blob('secrets.cfg.encrypted')
    blob.download_to_filename(encrypted_file_name)

akiatoji

import os
import uuid
from google.cloud import storage, kms_v1


def load(app):
    config_file_name = "secrets-%s.cfg" % str(uuid.uuid4())

    storage_client = storage.Client()
    bucket = storage_client.get_bucket('vault')

akiatoji

from my_lib import secrets

app = Flask(__name__)

# Load default config
app.config.from_pyfile('config/default_settings.py')

# Overlay secure secrets
secrets.load(app)