Skip to content

Instantly share code, notes, and snippets.

View matwerber1's full-sized avatar

Mathew Werber matwerber1

  • Seattle
View GitHub Profile
@matwerber1
matwerber1 / ec2-node-amzn.sh
Created September 26, 2018 02:57 — forked from ogckw/ec2-node-amzn.sh
ec2-amazon-linux-node-userdata
#!/bin/bash
# Program:
# EC2 initially install node.js, git for development environment.
# You can modify nodev and nvmv for changing node and nvm version.
# Set permission to ec2-user install above.
# History:
# 2017/07/25 Hans First release
home=/home/ec2-user
nodev='8.11.2'
nvmv='0.33.11'
@matwerber1
matwerber1 / google-sheet-formula-parse-iso86001-date-strings.md
Created July 30, 2022 03:28
Google Sheet Formula to convert ISO86001 date from string to usable timestamp data type.

Google sheet formula to convert a ISO86001 string to a date time:

Input: 2022-06-23T16:31:34.036000+00:00 (cell A2)

Formula: SUM(SPLIT(left(A2,len(A2)-6),"T"))

Output: (a properly-formatted datetime)

@matwerber1
matwerber1 / athena-query-for-cloudtrail-route53-dns-changes.sql
Last active October 15, 2023 04:42
Example SQL to search CloudTrail Logs stored in S3 using Amazon Athena for any changes to Route 53 DNS records#
-- Query assumes your Athena table is partitioned with a "day" column as described here:
-- https://www.linkedin.com/pulse/using-athena-partition-projection-query-cloudtrail-other-kinsman/
with detail AS (
SELECT
day,
eventtime as timestamp,
account,
awsregion as region,
eventid,
@matwerber1
matwerber1 / aws-cli-ecs-exec-with-fargate.sh
Last active October 13, 2023 04:18
Example to start an ECS Fargate task an existing cluster, task definition, and VPC, and then use ECS Exec to open an interactive shell with a containr in the task
CLUSTER="your_cluster_name"
REGION="cluster_region"
PRIVATE_SUBNETS="subnet-someSubnet123,subnet-someOtherSubnet456"
TASK_SECURITY_GROUP="security-group-for-ECS-task-ID"
PUBLIC_IP_SETTING="DISABLED"
TASK_CONTAINER_NAME_FOR_ECS_EXEC="container-name-for-ecs-exec"
# Run a Fargate task with ECS Exec enabled:
RUN_TASK_RESULT=$(
aws ecs run-task \
@matwerber1
matwerber1 / bigquery-sql-for-gmail-logs.sql
Last active October 19, 2023 12:36
Example SQL for Gmail Logs data written to BigQuery daily_ table
-- I was helping a company using Google Workspace for email, and they were having challenges
-- with email deliverability due to domain reputation. While it's true that gmail provides
-- provides detailed logs, I found their docs to be only marginaly helpful and lacking
-- more robust, real-world ways to interpret the data. I also struggled to find quality examples
-- when searching unofficial sources (blogs, etc.).
--
-- The query below is my best effort attempt to translate gmail logs into something half-way usable.
-- I'd be shocked if there weren't mistakes or areas for improvement, but either way, sharing in
-- the hope that it at least helps give a jumping-off point for anyone else that finds themselves
-- saying "OK, I enabled Gmail Logs for BigQuery.... now what?".
@matwerber1
matwerber1 / javascript-regex-for-aws.md
Last active October 13, 2023 04:16
Javascript regex patterns for AWS

AWS Resource Regex Patterns

AWS Backup

These were extracted from the open source AWS CLI code on GitHub for AWS Backup:

ruleName: /^[a-zA-Z0-9\-_\.]{1,50}$/,
@matwerber1
matwerber1 / requirements.txt
Created September 22, 2023 01:01
cloudmapper-python-requirements-for-m1-arm
astroid==2.15.6
autoflake==2.2.1
autopep8==2.0.4
boto3==1.28.52
botocore==1.31.52
certifi==2023.7.22
chardet==5.2.0
charset-normalizer==3.2.0
contourpy==1.1.1
coverage==7.3.1
@matwerber1
matwerber1 / gist:48ecc538ce03ce3d1708121bd34eba80
Last active October 17, 2023 20:32
Cleaning up my gists, found this. Not sure if it works, but making it public in case it helps.
/* eslint-disable no-prototype-builtins */
var express = require('express');
var ip = require("ip");
var fetchTimeout = require('fetch-timeout');
/*
This app creates a simple HTTP listener and responds to GET / requests by
displaying basic info about the container (e.g. IP address) and, optionally,
ECS metadata if the container is running on EC2 or Fargate via ECS.
@matwerber1
matwerber1 / aws-cli-eksctl-create-iamidentitymapping
Last active October 13, 2023 04:06
Snippet of Amazon EKS aws-auth configmap that grants cluster admin permissions to a specific AWS SSO role/permission set
# Example command
eksctl create iamidentitymapping \
--cluster YOUR_CLUSTER_NAME \
--arn arn:aws:iam::999999999999:role/AWSReservedSSO_YOUR-ROLE_NAME_xxxxxxxxxxxc \
--username cluster-admin \
--group system:masters

CloudTrail Lake Snippets

CloudTrail Lake supports Presto SQL. This doc focuses less on SQL itself and more on snippets specific to CloudTrail Lake schema.

Example Query

This query ties together key concepts and, in one shape or another, is often the starting point for my analyses of events:

SELECT