This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
""" | |
This script reads and processes Puffin files, extracting and printing blob metadata. | |
It supports reading metadata from the file footer and decompressing the | |
data using the specified compression codec. | |
Spec: | |
https://iceberg.apache.org/puffin-spec/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
AWSTemplateFormatVersion: '2010-09-09' | |
Description: A CloudFormation template to create IAM role, policy, Step Functions, and CloudWatch event rule and target for table ingestion maintenance vacuum with concurrency control | |
Parameters: | |
WorkgroupName: | |
Type: String | |
Default: primary | |
Description: The name of the Athena Workgroup | |
DatabaseName: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
provider "aws" { | |
region = "us-west-2" | |
} | |
resource "aws_s3_bucket" "my_bucket" { | |
bucket = "unique-bucket-name" | |
} | |
resource "aws_s3_access_point" "my_access_point" { | |
name = "myaccesspoint" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Create scratch location | |
cd | |
mkdir wsl-ubuntu-24.04 | |
# Download WSL image | |
wget ` | |
https://cloud-images.ubuntu.com/wsl/noble/current/ubuntu-noble-wsl-amd64-24.04lts.rootfs.tar.gz ` | |
-O ubuntu-noble-wsl-amd64-24.04lts.rootfs.tar.gz | |
# Import image to new WSL VM |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
""" | |
This script captures images from an RTSP stream at regular intervals and uses | |
the Claude 3 Haiku model to generate descriptions based on a custom prompt. | |
Install: | |
python3 -m venv .venv | |
source .venv/bin/activate | |
pip3 install boto3==1.34.62 opencv-python-headless==4.9.0.80 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
# | |
# -- Run other script to create the Iceberg table | |
# | |
# pip install boto3 | |
# curl https://gist.githubusercontent.com/t04glovern/04f6f2934353eb1d0fffd487e9b9b6a3/raw \ | |
# > lets-try-iceberg.py \ | |
# && chmod +x lets-try-iceberg.py | |
# ./lets-try-iceberg.py --table lets_try_iceberg | |
# |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
# | |
# pip install python-frontmatter openai | |
# ./spell-check-post.py content/blog/amazon-eventbridge-slack-notification-on-event/index.mdx | |
# | |
# The following file will be created: | |
# content/blog/amazon-eventbridge-slack-notification-on-event/index.new.mdx | |
# | |
# Follow the stream of content: | |
# tail -f content/blog/amazon-eventbridge-slack-notification-on-event/index.new.mdx |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Take the text after the "clippy" command as the input | |
user_prompt="$*" | |
export HELPER="Return only the shell/bash command for the following request, only return the bash/shell that would work if passed to the shell/bash. nothing more. example could be 'list all the files in my home directory' which would return 'ls ~' and ONLY the command 'ls ~', absolutely no other text around it, just the command - the request is for for the following:" | |
# Send the POST request and process the response | |
response="" | |
is_done=false |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
""" | |
This script generates sample data, uploads it to an S3 bucket, and creates Iceberg and Athena tables. | |
It also creates IAM roles and policies for optimization and statistics generation if specified. | |
There is also an option to create a Firehose delivery stream and insert random records into it. | |
Install: | |
python3 -m venv .venv | |
source .venv/bin/activate |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# | |
# pip install boto3 | |
# ./partition-projection-synthetic-data.py --bucket <bucket-name> | |
# | |
# Creates the following files: | |
# - partition-projection-create-table.sql - CREATE TABLE statement | |
# - partition-projection-query-table.sql - SQL to query the table | |
import logging |
NewerOlder