Skip to content

Instantly share code, notes, and snippets.

View alukach's full-sized avatar
🍊

Anthony Lukach alukach

🍊
View GitHub Profile
@alukach
alukach / ecs-rds-connector.py
Last active February 12, 2024 17:12
AWS Helper Scripts
View ecs-rds-connector.py
"""
This script provides a CLI to select an AWS ECS Service and multiple RDS Instances
and makes the required Security Group edits to allow the ECS Service to make network
connections to the RDS Instances
"""
from typing import List, Dict
import boto3
from botocore.exceptions import ClientError
@alukach
alukach / create-contractor-accounts.sh
Last active February 5, 2024 20:05
Script to create a contractor group and multiple users on an AWS account
View create-contractor-accounts.sh
#!/bin/bash
# Check if at least two arguments are provided (group name and at least one user)
if [ "$#" -lt 2 ]; then
echo "Usage: $0 <GroupName> <User1> [<User2> ...]"
exit 1
fi
# The first argument is the group name
GROUP="$1"
@alukach
alukach / list-bucket-metrics.py
Created January 30, 2024 20:29
List bytes and objects per storage type for all buckets in an AWS account
View list-bucket-metrics.py
import boto3
import csv
import threading
from datetime import datetime, timedelta
# List of storage types
storage_types = [
"StandardStorage",
"IntelligentTieringFAStorage",
@alukach
alukach / multiprocessing_asyncio.py
Created May 4, 2023 05:21
Multiprocessing + Asyncio
View multiprocessing_asyncio.py
"""
An example of a script that does CPU-bound work (checksum calculation) followed by
IO-bound work (upload to server) in a performant manner.
Inspiration: https://stackoverflow.com/questions/21159103/what-kind-of-problems-if-any-would-there-be-combining-asyncio-with-multiproces#29147750
"""
import asyncio
import datetime
import hashlib
import multiprocessing
@alukach
alukach / should-i-use-a-nat.md
Last active April 26, 2023 07:32
Should I use a NAT in my project?
View should-i-use-a-nat.md

A quick dump of criteria for deciding whether your project needs a NAT and, if so, what type it should be.

graph TD
    A(Do you need a NAT?) --> B
    B{Do you have services in a Private Subnet that\nneed to access resources outside of the network?}
    
    B -->|No| NotNeeded[You don't need a NAT]
    B -->|Yes| C
    C{Can you move those resources in a Public Subnet?} -->|Yes| PublicSubnet[Move to a Public Subnet instead] --> NotNeeded
@alukach
alukach / app.yaml
Last active February 2, 2024 05:55
An example Github Actions for Python + Pipenv + Postgres + Pyright
View app.yaml
# .github/workflows/app.yaml
name: My Python Project
on: push
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 10
services:
@alukach
alukach / boilerplate.py
Last active December 23, 2021 06:58
AWS S3 Batch Operation boilerplate
View boilerplate.py
import urllib
import boto3
from botocore.exceptions import ClientError
s3 = boto3.resource("s3")
TMP_FAILURE = "TemporaryFailure"
FAILURE = "PermanentFailure"
@alukach
alukach / parse-inventory-progress.py
Last active October 6, 2023 18:40
Parsing S3 Inventory results in Python
View parse-inventory-progress.py
#! /usr/bin/env python3
"""
A utility to stream records from one or many S3 Inventory reports, with a progress bar.
./parse-inventory-progress s3://my-bucket/path/to/my/inventory/2019-12-15T00-00Z/manifest.json > out.csv
"""
import json
import csv
import gzip
import sys
@alukach
alukach / example.py
Last active March 14, 2020 19:52
An example of a PIL-friendly object to represent an S3 file.
View example.py
from typing import IO
from dataclasses import dataclass, field
from io import BytesIO
import boto3
from PIL import Image
s3 = boto3.resource("s3")
@alukach
alukach / bundles.json
Last active November 6, 2019 04:50
Convert Planet product bundles description into JSON
View bundles.json
{
"bundles": {
"analytic": {
"name": "Analytic Radiance",
"description": "Orthorectified product, calibrated to at-sensor radiance",
"assets": {
"Landsat8L1G": [
"analytic_b1",
"analytic_b2",
"analytic_b3",