Skip to content

Instantly share code, notes, and snippets.

@devgrok
devgrok / DeduplicationKeyValueTransformerIntegrationTest.java
Last active Feb 4, 2021
Deduplicating in kafka-streams using the Processor API
View DeduplicationKeyValueTransformerIntegrationTest.java
/*
* Copyright Confluent Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
@devgrok
devgrok / manifest-uri-go-getter.patch
Created Feb 19, 2020
fix 'unsupported protocol scheme "git"' in latest Kfctl
View manifest-uri-go-getter.patch
Index: kfctl/pkg/kfconfig/types.go
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- kfctl/pkg/kfconfig/types.go (revision 94c35cfa53235961a329cefd144ecf524fe5cea4)
+++ kfctl/pkg/kfconfig/types.go (date 1582095567552)
@@ -11,9 +11,11 @@
"os"
"path"
@devgrok
devgrok / minio-ec2-iam.patch
Created Jan 10, 2020
fixes for minio IamEc2MetaData
View minio-ec2-iam.patch
Index: .github/workflows/pythonpackage-linux.yml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- .github/workflows/pythonpackage-linux.yml (revision 3331f178884ac36da622a81e73ab981e25183783)
+++ .github/workflows/pythonpackage-linux.yml (date 1578582406638)
@@ -26,7 +26,7 @@
python-version: ${{ matrix.python-version }}
- name: Install dependencies
@devgrok
devgrok / README.md
Last active Nov 23, 2019
Set profile-based AWS environment variables with support for assume-role
View README.md

About

This script sets AWS_DEFAULT_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN (where applicable) from the values configured in ~/.aws/config and ~/.aws/credentials This is useful for cases where the tool being used doesn't support --profile being passed in, i.e. most unit/system tests

It also can handle cases where the credentials aren't static (i.e. assume-role giving temporary credentials), it uses the awscli's implementation to resolve and cache those credentials. This is very handy for when a tool is using the aws-sdk to load credentials and can read the AWS_PROFILE environment variable but can't handle the assume-role, usuaully due to an MFA prompt. In those cases it would reolve the tokens and set them as env vars. Calling it again would refersh them if required.

@devgrok
devgrok / amazon-linux-1-user-data.txt
Last active May 12, 2020
Add ephemera storage to Amazon ECS-optimized AMIs (v1)
View amazon-linux-1-user-data.txt
Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0
--==BOUNDARY==
MIME-Version: 1.0
Content-Type: text/cloud-config
# this script ensures that empheral storage doesn't get mounted by cloud-init
mounts:
- [ ephemeral0 ]
@devgrok
devgrok / hipchat_proxy.py
Last active Mar 21, 2019
Create a Private Microservice Using an Application Load Balancer
View hipchat_proxy.py
import json
import logging
import os
from lambdarest import lambda_handler
from slackclient import SlackClient
logger = logging.getLogger()
logger.setLevel(logging.INFO)
@devgrok
devgrok / cloud-init-cloudwatch-agent
Created Jul 2, 2018
Installs CloudWatch Agent for monitoring ECS-Agent log files
View cloud-init-cloudwatch-agent
Content-Type: multipart/mixed; boundary="==BOUNDARY=="
MIME-Version: 1.0
--==BOUNDARY==
MIME-Version: 1.0
Content-Type: text/cloud-boothook; charset="us-ascii"
#!/bin/bash +x
# optionally set ECS agent options (increase base container size)
cloud-init-per once docker_options echo 'OPTIONS="${OPTIONS} --storage-opt dm.basesize=100G"' >> /etc/sysconfig/docker