Skip to content

Instantly share code, notes, and snippets.

View timoguin's full-sized avatar
🚫
No

Tim O'Guin timoguin

🚫
No
73 followers · 418 following
View GitHub Profile
@timoguin
timoguin / step01-create_manifest.sh
Last active March 7, 2024 02:30
S3 Batch Job that copies objects from a source to a destination bucket
#!/bin/sh
# List objects in an S3 bucket and generate a CSV manifest suitable for
# driving an S3 Batch Job. Works for smol buckets. Just pass a valid S3 URL.
#
# Usage:
#
# ./create_manifest.sh s3://muh-buckit/muh-prefix/
#
NUM_ARGS=$#
@timoguin
timoguin / gist:2000f64e7227ce9e1271de01980f4041
Created February 22, 2024 23:17
keybase.md
### Keybase proof
I hereby claim:
* I am timoguin on github.
* I am timoguin (https://keybase.io/timoguin) on keybase.
* I have a public key ASASOw1l4f5hOtmgIXRXcpAqiydqiYFZZgykaewaEpvvqwo
To claim this, I am signing this object:
@timoguin
timoguin / update_issues.sh
Created February 4, 2024 22:36
Bulk add GitHub Issues to a GitHub Project
set -ex
# Use the GitHub CLI to loop through all repositories in an organization, get all the
# open issues, and add them to a single org-level project.
#
# Note: This was originally done with GNU Parallel but was refactored with a slower for
# loop due to hitting GitHub rate limits.
# Loop through list of repos
for repo in $(gh repo list tiki --json owner,name --jq '.[] | (.owner.login + "/" + .name)'); do
@timoguin
timoguin / sparky_spark.py
Created January 24, 2024 23:01
Working with Iceberg tables in Athena Notebooks (pySpark)
from pyspark.sql import DataFrame
import boto3
GLUE_CLIENT = boto3.client("glue", region_name="us-east-2")
def get_table_metadata_location(glue_client, table: str) -> str:
"""
@timoguin
timoguin / data_dynamodb_query.tf
Created February 27, 2020 15:47
Terraform AWS Provider: Proposed design for DynamoDB Query and GetItem Data Sources
# =========================================================================== #
#
# Proposed arguments for DynamoDB query data source, using an HCL example.
#
# Status: Initial Design
#
# --------------------------------------------------------------------------- #
#
# We would want to stick pretty closely to DynamoDB's Query API for this:
# https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html
@timoguin
timoguin / iam.tf
Created June 16, 2017 18:41
AWS Require MFA IAM Policy attached to all users
# IAM policy for all users that requires MFA
data "aws_iam_policy_document" "require_mfa_policy" {
statement {
sid = "AllowAllUsersToListAccounts"
effect = "Allow"
actions = [
"iam:ListAccountAliases",
"iam:ListUsers",
]
@timoguin
timoguin / policy.json
Created December 28, 2016 16:38
IAM policy for the Packer Builder
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "NonResourceBasedReadOnlyPermissions",
"Effect": "Allow",
"Action": [
"ec2:DescribeVolumes",
"ec2:DescribeKeyPairs",
"ec2:DescribeSecurityGroups",
@timoguin
timoguin / gist:51778e6ecd64ca5ab775
Created September 25, 2014 16:38
salt managed file with dynamic filename
{% set files_list = salt['file.find']('/path/', name='*pattern*') %}
{% if files_list|length > 0 %}
{% set filename = files_list[0] %}
{% endif %}
ensure managed file:
file.managed:
- name: {{ filename }}
@timoguin
timoguin / gist:9964865
Created April 3, 2014 23:23
cmd.run module vs. cmd.run state from the command line
salt@master01:~$ salt web01.flexdeaf.dev cmd.run 'ufw allow proto tcp from 192.168.1.115 to any port 9997'
web01.flexdeaf.dev:
Rules updated
salt@master01:~$ salt web01.flexdeaf.dev state.single cmd.run 'ufw allow proto tcp from 192.168.1.115 to any port 9998'
web01.flexdeaf.dev:
----------
ID: ufw allow proto tcp from 192.168.1.115 to any port 9998
Function: cmd.run
Result: True
Comment: Command "ufw allow proto tcp from 192.168.1.115 to any port 9998" run
@timoguin
timoguin / .kitchen.yml
Created March 6, 2014 20:27
snippet for specifying box in .kitchen.yml
platforms:
- name: ubuntu-12.04
driver_config:
box: ubuntu-12.04-cloudimg
box_url: http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-vagrant-amd64-disk1.box