Skip to content

Instantly share code, notes, and snippets.

View devgrok's full-sized avatar

Chris Watts devgrok

10 followers · 0 following
View GitHub Profile
@devgrok
devgrok / DeduplicationKeyValueTransformerIntegrationTest.java
Last active October 2, 2024 13:18
Deduplicating in kafka-streams using the Processor API
/*
* Copyright Confluent Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@devgrok
devgrok / manifest-uri-go-getter.patch
Created February 19, 2020 07:04
fix 'unsupported protocol scheme "git"' in latest Kfctl
Index: kfctl/pkg/kfconfig/types.go
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- kfctl/pkg/kfconfig/types.go (revision 94c35cfa53235961a329cefd144ecf524fe5cea4)
+++ kfctl/pkg/kfconfig/types.go (date 1582095567552)
@@ -11,9 +11,11 @@
"os"
"path"
@devgrok
devgrok / minio-ec2-iam.patch
Created January 10, 2020 03:00
fixes for minio IamEc2MetaData
Index: .github/workflows/pythonpackage-linux.yml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- .github/workflows/pythonpackage-linux.yml (revision 3331f178884ac36da622a81e73ab981e25183783)
+++ .github/workflows/pythonpackage-linux.yml (date 1578582406638)
@@ -26,7 +26,7 @@
python-version: ${{ matrix.python-version }}
- name: Install dependencies
@devgrok
devgrok / README.md
Last active November 23, 2019 04:50
Set profile-based AWS environment variables with support for assume-role

About

This script sets AWS_DEFAULT_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN (where applicable) from the values configured in ~/.aws/config and ~/.aws/credentials This is useful for cases where the tool being used doesn't support --profile being passed in, i.e. most unit/system tests

It also can handle cases where the credentials aren't static (i.e. assume-role giving temporary credentials), it uses the awscli's implementation to resolve and cache those credentials. This is very handy for when a tool is using the aws-sdk to load credentials and can read the AWS_PROFILE environment variable but can't handle the assume-role, usuaully due to an MFA prompt. In those cases it would reolve the tokens and set them as env vars. Calling it again would refersh them if required.

@devgrok
devgrok / amazon-linux-1-user-data.txt
Last active January 25, 2022 09:53
Add ephemera storage to Amazon ECS-optimized AMIs (v1)
Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0
--==BOUNDARY==
MIME-Version: 1.0
Content-Type: text/cloud-config
# this script ensures that empheral storage doesn't get mounted by cloud-init
mounts:
- [ ephemeral0 ]
@devgrok
devgrok / hipchat_proxy.py
Last active March 21, 2019 17:09
Create a Private Microservice Using an Application Load Balancer
import json
import logging
import os
from lambdarest import lambda_handler
from slackclient import SlackClient
logger = logging.getLogger()
logger.setLevel(logging.INFO)
@devgrok
devgrok / cloud-init-cloudwatch-agent
Created July 2, 2018 11:32
Installs CloudWatch Agent for monitoring ECS-Agent log files
Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0
--==BOUNDARY==
MIME-Version: 1.0
Content-Type: text/cloud-boothook; charset="us-ascii"
#!/bin/bash +x
# optionally set ECS agent options (increase base container size)
cloud-init-per once docker_options echo 'OPTIONS="${OPTIONS} --storage-opt dm.basesize=100G"' >> /etc/sysconfig/docker