anoasis

Kafka Topics

List existing topics

kafka-topics --zookeeper localhost:2181 --list

Describe a topic

kafka-topics --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

kafka-topics --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

anoasis

package com.github.stepancheg.protobuftojson;

import com.github.stepancheg.protobuftojson.argv.ArgvReader;
import com.github.stepancheg.protobuftojson.io.FileUtils;
import com.github.stepancheg.protobuftojson.protobuf.ProtobufUtils;
import com.google.protobuf.DescriptorProtos;
import com.google.protobuf.Descriptors;
import com.google.protobuf.DynamicMessage;
import com.google.protobuf.TextFormat;

anoasis

https://mhausenblas.info/kube-rpi/#:~:text=Install%20Helm,-First%20we%20need&text=Copy%20the%20content%20of%20%2Fetc,%2Fetc%2Fhosts%20file%20%3B)
https://ubuntu.com/tutorials/how-to-install-ubuntu-on-your-raspberry-pi#1-overview

- SSH to Node -
nmap -sP 192.168.1.0/24


- Install Micro K8S -
https://ubuntu.com/tutorials/how-to-kubernetes-cluster-on-raspberry-pi#1-overview

anoasis

# count number of message per topic
kafka-run-class kafka.admin.ConsumerGroupCommand --group <consumer-group> --bootstrap-server localhost-0:9092 --describe --command-config /bin/connect.propertiesscribe
(deprecated)kafka-run-class.sh kafka.tools.ConsumerOffsetChecker  --topic test --zookeeper localhost:2181 --group testgroup
kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic <topic-name> --time -1 --offsets 1 
  | awk -F  ":" '{sum += $3} END {print sum}'

anoasis

set -eux

declare -r HOST="http://google.com"

wait-for-url() {
    echo "Testing $1"
    timeout -s TERM 45 bash -c \
    'while [[ "$(curl -s -o /dev/null -L -w ''%{http_code}'' ${0})" != "200" ]];\
    do echo "Waiting for ${0}" && sleep 2;\
    done' ${1}

anoasis

from __future__ import print_function

import json

import boto3
import urllib.request

ec2_client = boto3.client('ec2', region_name='us-west-1')
ssm_client = boto3.client('ssm', region_name='us-west-1')

anoasis

            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <filters>
                <filter>
                  <!-- Do not copy the signatures in the META-INF folder.
                  Otherwise, this might cause SecurityExceptions when using the JAR. -->

anoasis

import inspect
attr = [i for i in dir(class) if not inspect.ismethod(i)]
attr

anoasis

#!/bin/bash
find $1 -type f -exec stat --format '%Y :%y %n' "{}" \; | sort -nr | cut -d: -f2- | head

anoasis

Prometheus components do not provide a built-in way tosecure their interfaces in any way, at least for now. If no additional components are set up, which would enable encryption or authentication (or both), all the traffic between Prometheus and its components is sent in plain text, and there are no access restrictions - anyone, who knows where to look, can access these interfaces.

Securing this setup would usually involve some sort of reverse proxy in front of Prometheus and its components, which could enable both - traffic encryption with a TLS certificate, and also authentication with, for example, username and password or the same TLS certificate. There are numerous options out there for this purpose - nginx, HAproxy, hitch, ghostunnel etc.

This post would go into a bit more technical details on how to secure communication between Prometheus and node_exporter on a remote system with the help of TLS certificates and one such tool - stunnel.

TLS certificates
First off, we will need TLS certificates for a