rmoff

Configuring Kafka Connect with Confluent Cloud

So you want to connect Kafka Connect to Confluent Cloud? Here’s how.

Overview

1. Configure the worker

2. Pre-create any topics that your source connector will write to (since auto-topic creation is not enabled on Confluent Cloud)

bradwilson

{
    "$help": "https://aka.ms/terminal-documentation",
    "$schema": "https://aka.ms/terminal-profiles-schema",
    "actions": [
        {
            "command": "copy",
            "keys": "ctrl+shift+c"
        },
        {
            "command": "paste",

amark

<script>
	$.as.route.page('settings', () => {
		if(!S.user.is){ return $.as.route('hi') }
		$.as('#settings', S.user);
		
		;(() => {
			$('#pet').on('keyup', function(){ $('.pet').text(this.value.length + ' letters.') });
			$('#kiss').on('keyup', function(){ $('.kiss').text(this.value.length + ' letters.') });
			
			S.user.get('settings').get('pet').on(function(l){ $('#pet').val(Gun.text.random(l, '*')) });

tedivm

#!/usr/bin/env bash

apt_wait () {
  while sudo fuser /var/lib/dpkg/lock >/dev/null 2>&1 ; do
    sleep 1
  done
  while sudo fuser /var/lib/apt/lists/lock >/dev/null 2>&1 ; do
    sleep 1
  done
  if [ -f /var/log/unattended-upgrades/unattended-upgrades.log ]; then

sahilsk

Kafka Cheat Sheet

Display Topic Information

$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic beacon
Topic:beacon	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: beacon	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	Topic: beacon	Partition: 1	Leader: 1	Replicas: 1	Isr: 1

chrisveness

/**
 * Encrypts plaintext using AES-GCM with supplied password, for decryption with aesGcmDecrypt().
 *                                                                      (c) Chris Veness MIT Licence
 *
 * @param   {String} plaintext - Plaintext to be encrypted.
 * @param   {String} password - Password to use to encrypt plaintext.
 * @returns {String} Encrypted ciphertext.
 *
 * @example
 *   const ciphertext = await aesGcmEncrypt('my secret text', 'pw');

ursuad

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

HR

'use strict'
/**
 * AES file encryption with authentication (using Promises)
 * (C) Habib Rehman 2016
 ******************************/
 
const encrypt = function (origpath, destpath, key) {
  // decrypts any arbitrary data passed with the pass
  return new Promise(function (resolve, reject) {
    // readstream to read the (unencrypted) file

harlow

Golang Workers / Job Queue

A running example of the code from:

• http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang
• http://nesv.github.io/golang/2014/02/25/worker-queues-in-go.html

This gist creates a working example from blog post, and a alternate example using simple worker pool.

TLDR: if you want simple and controlled concurrency use a worker pool.

AndiDittrich

// SPDX-License-Identifier: MPL-2.0

// AES Encryption/Decryption with AES-256-GCM using random Initialization Vector + Salt
// ----------------------------------------------------------------------------------------
// the encrypted datablock is base64 encoded for easy data exchange. 
// if you have the option to store data binary save consider to remove the encoding to reduce storage size
// ----------------------------------------------------------------------------------------
// format of encrypted data - used by this example. not an official format
//
// +--------------------+-----------------------+----------------+----------------+