Skip to content

Instantly share code, notes, and snippets.

@jpbarto
jpbarto / region_network_stats.py
Last active March 26, 2018 23:42
Retrieve the ingress and egress network metrics for all EC2 instances in a given AWS account and region
#!/usr/bin/python
from datetime import datetime, timedelta
import boto3
# region to query and what AWS credentials profile to use
REGION = 'eu-west-2'
PROFILE = 'default'
MONTH_PERIOD = 3600 * 24 * 30 # ask CloudWatch to average over a 30 day
@jpbarto
jpbarto / readme.md
Last active July 17, 2018 09:21
Light speed on AWS

the following estimates the theoretical network latency for travelling roughly half way around the earth

circumference of earth = 40075 km

speed of light = 299792.458 km / s

( circumference of earth ) / 2 ) / ( speed of light ) = seconds to travel from one side of globe to another

(40075 / 2) / (299792.458) = 0.0668 seconds

67 ms ideal travel time from one side of globe to another

@jpbarto
jpbarto / websocket-nlb-test.tf
Created March 16, 2019 21:57
A Terraform template to create an AWS autoscaling group tied to a network load balancer, to test the ability to create a Socket.IO server that has no route to the Internet
##
## AMI created using
## curl -sL https://rpm.nodesource.com/setup_11.x | bash -
## yum install -y git nodejs
## git clone https://github.com/socketio/socket.io.git
##
## cd socket.io
## npm install
## cd examples/chat
## npm install
@jpbarto
jpbarto / video-game-sales.ipynb
Created March 26, 2019 16:17
Sample Jupyter Notebook
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@jpbarto
jpbarto / sagemaker_training_job.json
Created June 20, 2019 11:19
Sample Script Container Invoke
{
"TrainingJobName": "cli-t-job",
"HyperParameters": {
"sagemaker_submit_directory": "s3://sagemaker-eu-west-2-776347453069/sagemaker-scikit-learn-2019-06-19-13-21-02-471/source/sourcedir.tar.gz",
"sagemaker_program": "sklearn_abalone_featurizer.py",
"sagemaker_enable_cloudwatch_metrics": "false",
"sagemaker_container_log_level": "20",
"sagemaker_job_name": "sagemaker-scikit-learn-2019-06-19-13-21-02-471",
"sagemaker_region": "eu-west-2"
},
@jpbarto
jpbarto / private-eks-nodes.yaml
Created July 29, 2019 06:44
Private EKS Worker Nodes - CloudFormation
###
#
# CloudFormation Template to create EKS Worker Nodes without access to an IGW
#
# The following CloudFormation template is based upon the AWS-provided CloudFormation template for creating
# EKS worker nodes in your VPC. It differs in that it has user-data modified to configure a node to operate
# within a VPC that has no IGW.
#
# To use this template you will need a VPC with an EKS cluster already deployed and private access enabled.
# You will also need a web proxy configured to allow the worker nodes access to the AWS EKS service API. This
@jpbarto
jpbarto / private-eks-workers.yaml
Created August 18, 2019 06:44
Private EKS Workers CloudFormation with User Data
AWSTemplateFormatVersion: 2010-09-09
Description: Amazon EKS - Node Group
Parameters:
KeyName:
Description: The EC2 Key Pair to allow SSH access to the instances
Type: 'AWS::EC2::KeyPair::KeyName'
NodeImageId:
Description: AMI id for the node instances.
Type: 'AWS::EC2::Image::Id'
NodeInstanceType:
@jpbarto
jpbarto / fashionmnist-mxnet.py
Last active May 8, 2020 19:40
Sample MXNet code for Fashion MNIST
"""
This script is a simple FashionMNIST training script which uses MXNet's.
It has been orchestrated with SageMaker Debugger hook to allow saving tensors during training.
Here, the hook has been created using its constructor to allow running this locally for your experimentation.
When you want to run this script in SageMaker, it is recommended to create the hook from json file.
Please see scripts in either /examples/tensorflow/sagemaker_byoc or /examples/tensorflow/sagemaker_official_container
folder based on your use case.
"""
# Standard Library
import argparse
@jpbarto
jpbarto / ide-lifecycle.sh
Last active May 15, 2020 20:03
Lifecycle script for SageMaker notebook startup which installs Theia-IDE
#!/bin/bash
set -e
sudo -u ec2-user -i <<'EOP'
## INSTALL THEIA IDE FROM SOURCE
EC2_HOME=/home/ec2-user
mkdir ${EC2_HOME}/theia && cd ${EC2_HOME}/theia
### begin by installing NVM, NodeJS v10, and Yarn
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.5/install.sh | bash
@jpbarto
jpbarto / workshops-worth-shopping.md
Created May 29, 2020 21:52
Collection of Workshops worth working through