Skip to content

Instantly share code, notes, and snippets.

View cschultz82's full-sized avatar
🎯
Focusing

cschultz82

🎯
Focusing
View GitHub Profile
@jitsejan
jitsejan / _write_dataframe_to_csv_on_s3.py
Last active February 13, 2024 12:53
Write a Pandas dataframe to CSV format on AWS S3.
import boto3
from io import StringIO
def _write_dataframe_to_csv_on_s3(dataframe, filename):
""" Write a dataframe to a CSV on S3 """
print("Writing {} records to {}".format(len(dataframe), filename))
# Create buffer
csv_buffer = StringIO()
# Write dataframe to buffer
dataframe.to_csv(csv_buffer, sep="|", index=False)
@lukeplausin
lukeplausin / bash_aws_jq_cheatsheet.sh
Last active January 4, 2025 17:42
AWS, JQ and bash command cheat sheet. How to query, cut and munge things in JSON generally.
# Count total EBS based storage in AWS
aws ec2 describe-volumes | jq "[.Volumes[].Size] | add"
# Count total EBS storage with a tag filter
aws ec2 describe-volumes --filters "Name=tag:Name,Values=CloudEndure Volume qjenc" | jq "[.Volumes[].Size] | add"
# Describe instances concisely
aws ec2 describe-instances | jq '[.Reservations | .[] | .Instances | .[] | {InstanceId: .InstanceId, State: .State, SubnetId: .SubnetId, VpcId: .VpcId, Name: (.Tags[]|select(.Key=="Name")|.Value)}]'
# Wait until $instance_id is running and then immediately stop it again
aws ec2 wait instance-running --instance-id $instance_id && aws ec2 stop-instances --instance-id $instance_id
# Get 10th instance in the account
@robbwagoner
robbwagoner / ebs-costs.py
Last active December 4, 2020 00:27
Get EBS costs for an AWS Region (us-east-1 and us-west-1, for now)
#!/usr/bin/env python2.7
"""
Get an approximate cost of EBS for a Region, differentiating used (attached)
vs. unused (available) EBS volumes, and their type (standard & gp2 vs. io1).
Copyright 2015 Robb Wagoner robb.wagoner@gmail.com
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at