nmagee

#!/bin/bash
apt-get update
apt-get install -y nginx jq

# IMDSv2 token-based access (recommended)
TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

# Get instance metadata using the token
INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
AZ=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/placement/availability-zone)

nmagee

Create EC2 Instance

• NAME: iam-testing-instance
• Ubuntu 24.04LTS image
• t3.micro instance type
• Select your keypair
• Create new security group (or use existing one) that allows port 22 from your current address

Once created, SSH into the instance and await next steps.

nmagee

name: container-build

on:
  push:
    branches: [ main ]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ACCOUNT/IMAGE-NAME        # replace these as needed
  IMAGE_TAG: 1.${{ github.run_number }} # GITHUB_RUN_NUMBER, so versions will be 1.1, 1.2, etc.

nmagee

#!/usr/bin/env python

# http://boto3.readthedocs.org/en/latest/reference/services/rds.html
import boto3
import sys, getopt

bucket = None
file_path = None
timeout_seconds = 3600

nmagee

#!/bin/sh

# This script creates a series of prompts to create an AMI of an EC2 instance from a few AWS profiles
# Comes with a fixed US-East-1 to US-West-2 migration path. Modify accordingly for your own needs.
# 09.05.2014 Neal Magee

set -e
clear
echo ""
read -p "   What AWS profile does this AMI belong to? (Enter 'default' to use that profile): " profile

nmagee

#!/bin/sh

# This script creates a series of prompts to create an AMI of an EC2 instance from a few AWS profiles
# Note that the YYYYmmdd- is added to the name of the AMI automatically

set -e
clear

# Check for AWS CLI tools
type aws >/dev/null 2>&1 || { echo >&2 "\n\n   I require the AWS CLI but it's not installed. Please visit http://aws.amazon.com/cli/ and install it. Aborting.\n"; exit 1; }

nmagee

#!/usr/local/bin/python3

# A dumb/simple script to retrieve the value of a SecretsManager secret

import boto3
import base64
from botocore.exceptions import ClientError

def get_secret():

nmagee

Lab: JSON Serialization and Deserialization with REST APIs

Estimated Time: 30-45 minutes
Difficulty: Intermediate
Prerequisites: Basic Python, HTTP requests, JSON concepts

Learning Objectives

By the end of this lab, you will be able to:

• Fetch data from REST APIs and deserialize JSON responses

nmagee

Data Windowing

With endless streaming sources, how do we generate analytics? Windowing is a meaningful way of aggregating data into useful chunks. A consumer operation, not a producer operation. Think of it as a group_by operation for streaming data.

GROUP BY - for data at rest

Note the two attached python scripts that use the NYC Taxi Data and group results by either day or hour (of a single day).

nmagee

prefect

Install

In a new virtual environment:

pip install prefect requests sympy

Server