Skip to content

Instantly share code, notes, and snippets.

View ystoneman's full-sized avatar

Yann Stoneman ystoneman

View GitHub Profile
@ystoneman
ystoneman / get-csv-contents.sh
Created December 11, 2022 03:17
How to get a CSV file bigger than 1MB from GitHub
# replace vnd.github+json with vnd.github.raw+json when a file is bigger than 1MB
curl \
-H "Accept: application/vnd.github.raw+json" \
-H "Authorization: Bearer TOKEN"\
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/ystoneman/hotel-bookings/contents/hotel_bookings.csv
@ystoneman
ystoneman / gist:118daaf54acb0aa19def0e86719fbab1
Created November 28, 2022 17:35
Make SQL into a 1-liner in VS Code on Mac
- Remove All Comments (with command palette)
- Control-J (on Mac)
@ystoneman
ystoneman / sample.py
Created June 19, 2022 23:19
Loop through ndjson
# Just a note to remember how I looped through rows in an ndjson from S3
from botocore.vendored import requests
import json
import boto3
from io import StringIO
s3 = boto3.resource('s3')
my_bucket = s3.Bucket(bucket)
obj = s3.Object(bucket, key)
@ystoneman
ystoneman / terminationprotection.sh.txt
Last active March 1, 2022 12:18 — forked from huevos-y-bacon/aws_ec2_termination_protection.md
Enable or disable EC2 instance "Termination Protection" via AWS CLI (shell)
# Loop through all EC2 instances (except spot and terminated) and enable termination protection
for I in $(aws ec2 describe-instances --query 'Reservations[*].Instances[?(InstanceLifecycle!=`spot` && InstanceState!=terminated)].[InstanceId]' --output text); do aws ec2 modify-instance-attribute --disable-api-termination --instance-id $I; done
# Loop through all EC2 instances (except spot and terminated) and disable termination protection
for I in $(aws ec2 describe-instances --query 'Reservations[*].Instances[?(InstanceLifecycle!=`spot` && InstanceState!=terminated)].[InstanceId]' --output text);
do aws ec2 modify-instance-attribute --no-disable-api-termination --instance-id $I;
done
@ystoneman
ystoneman / fn-if-example.yaml
Created February 27, 2022 21:09
An example of how you can conditionally create resources in CloudFormation using Fn::If or !If for short.
AWSTemplateFormatVersion: 2010-09-09
Parameters:
SnsTopicArn:
Type: String
Default: ''
Description: >-
A pre-existing SNS topic to send S3 notifications to.
If not provided, a new topic will be created.
MyEmail:
Type: String
@ystoneman
ystoneman / put_one_log_event.py
Created February 21, 2022 21:35
Lambda function that puts a hello-world log message into CloudWatch Logs, just to test connectivity and permissions.
import json
import boto3
import time
def lambda_handler(event, context):
LOG_GROUP='awesome-logs'
LOG_STREAM='test'
logs = boto3.client('logs')
logs.create_log_stream(logGroupName=LOG_GROUP, logStreamName=LOG_STREAM)
@ystoneman
ystoneman / kms_policy.json
Created February 12, 2022 02:13
KMS Policy WIth Permissions for CloudWatch Logs
{
"Id": "only-cloudwatch-logs",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:root"
},
Instance type Private IPv4 addresses per interface
t3.nano 2
t3.small 4
t3.medium 6
m5.large 10
t4g.large 12
c4.xlarge 15
d3.8xlarge 20
c5.4xlarge 30
c5.metal 50
Instance type Maximum network interfaces
t3.nano 2
t3.small 3
t3.xlarge 4
m5.4xlarge 8
m5.16xlarge 15
p4d.24xlarge 4x15
Number of vCPU Number of vNICs
2 or less 2
2 to 8 2 to 8
8 or more 8