Skip to content

Instantly share code, notes, and snippets.

Avatar
🏠
Working from home

Mohan Balasundaram tuxfight3r

🏠
Working from home
View GitHub Profile
@tuxfight3r
tuxfight3r / kcat.md
Last active Aug 25, 2021
KafkaCat configuration for AWS MSK
View kcat.md

KafkaCat Configuration for AWS MSK

Set the below environment variable with the following values

NOTE: Kafkacat is renamed to kcat recently and the config variable should be KCAT_CONFIG for version 1.7 onwards.

# you can export the variable or present the config with -F parameter for kafkacat
export KAFKACAT_CONFIG=/home/tools/persistent/kcat/kafkacat_config

Contents of kafkacat configuration

View gist:082ddafb892e3bbf00251b48782413a7
#Simple script to test all the pods liveness/readiness
#creates portforward and curls the endpoint for status
#clears the portforward at the end of the script
#uses bash v3 based implementation.
#Test only the pods which are up.
#!/bin/bash
aggregate_feed="tfeed"
@tuxfight3r
tuxfight3r / gist:b67e94e5a09c61c0c31b00ec0a6e8eb5
Created Jul 14, 2021
python decode kubernetes service token (JWT Token)
View gist:b67e94e5a09c61c0c31b00ec0a6e8eb5
# retrieve the default kubernetes service account token
kubectl get secret $(kubectl get serviceaccounts default -o jsonpath='{.secrets[0].name}') -o json|jq -r .data.token |base64 -D
# Decode it using Python (pip install PyJWT)
import jwt
var="encoded.jwt.token"
jwt.decode(var, options={"verify_signature":False})
@tuxfight3r
tuxfight3r / selfsigned-clusterissuer.yml
Last active Jul 15, 2021
kube cert-manager cert issuer
View selfsigned-clusterissuer.yml
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: selfsigned-cluster-issuer
spec:
selfSigned: {}
@tuxfight3r
tuxfight3r / dialog.sh
Last active Jun 30, 2021
bash script to choose an option with dialog box
View dialog.sh
#!/bin/bash
HEIGHT=10
WIDTH=40
CHOICE_HEIGHT=4
BACKTITLE="Cluster Options"
TITLE="Select Cluster"
MENU="Choose one of the following options:"
ENV="${ENV:-dev}"
@tuxfight3r
tuxfight3r / kafka-cheat-sheet.md
Created May 7, 2021 — forked from ursuad/kafka-cheat-sheet.md
Quick command reference for Apache Kafka
View kafka-cheat-sheet.md

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

@tuxfight3r
tuxfight3r / notes.md
Last active Mar 17, 2021
validating EC2 pem / ssh key pairs
View notes.md
## Get private key fingerprint for AWS generated key ( will match the id in AWS Console)
$ openssl pkcs8 -in test_key.pem -inform PEM -outform DER -topk8 -nocrypt | openssl sha1 -c
(stdin)= f6:38:9c:53:9c:64:a8:b3:40:23:9f:6c:ed:0e:3d:bf:bf:16:bb:1c

## Get fingerpint from Openssh public key only md5 / sha1
$ ssh-keygen -f test_key.pub -e -m PKCS8 | openssl pkey -pubin -outform DER | openssl md5 -c
(stdin)= 55:b6:90:db:34:29:da:60:68:c9:08:24:f1:9f:c8:4b

(or)
@tuxfight3r
tuxfight3r / kubect_patch.md
Last active Mar 10, 2021
kubectl configmap json patch
View kubect_patch.md

use kubectl to patch configmap via json patch operations (add/remove/replace/test)

## Add a new key to the existing configmap
$ kubectl patch cm demo-app-config --type json --patch '[{ "op": "add", "path": "/data/TEST_VALUE", "value": "test_key" }]'
configmap/demo-app-config patched

$ kubectl get cm demo-app-config -o json | jq .data.TEST_VALUE
"test_key"
@tuxfight3r
tuxfight3r / kafka-cheat-sheet.md
Created Feb 24, 2021 — forked from sahilsk/kafka-cheat-sheet.md
Apache Kafka Cheat Sheet
View kafka-cheat-sheet.md

Kafka Cheat Sheet

Display Topic Information

$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic beacon
Topic:beacon	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: beacon	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	Topic: beacon	Partition: 1	Leader: 1	Replicas: 1	Isr: 1
@tuxfight3r
tuxfight3r / kafka-useful-commands.md
Created Feb 24, 2021 — forked from mjuric/kafka-useful-commands.md
Useful Kafka wrangling commands
View kafka-useful-commands.md

Utilities you'll care about

All these are already installed on epyc.

  • kafkacat (conda install -c conda-forge kafkacat)

  • kt (grab it from https://github.com/fgeller/kt/releases)

  • kafka-* (come with kafka, if you yum install if from Confluent's repo, or via Docker if you're so inclined). Warning -- JVM based and dreadfully slow.

  • jq (conda install -c conda-forge jq or use your favorite package manager)