MichaelDrogalis

I've been working with Apache Kafka for over 7 years. I inevitably find myself doing the same set of activities while I'm developing or working with someone else's system. Here's a set of Kafka productivity hacks for doing a few things way faster than you're probably doing them now. 🔥

• Show me all my Kafka topics and their partitions, replicas, and consumers
• Show me the contents of a topic
• Create a Kafka topic
• Produce messages to a Kafka topic
• Validate the schema of messages before producing to a topic
• Do all of this at a distance

Get the tools

rdump

kubectl multi-version brews

Applicability

The instructions below apply to older versions of Homebrew which still provide switch capability.

For current Homebrew, you'll likely need to keep Versions around, and build locally. Here's my versions repository https://github.com/rdump/homebrew-versions

MacPorts is now keeping versioned installations available as well, by default.

strongjz

Tools

• Kops https://github.com/kubernetes/kops/blob/master/docs/README.md
• Kubectl https://kubernetes.io/docs/tasks/tools/install-kubectl/
• kubespy https://blog.pulumi.com/kubespy-trace-a-real-time-view-into-the-heart-of-a-kubernetes-service
• kops https://icicimov.github.io/blog/virtualization/Kubernetes-Cluster-in-AWS-with-Kops/

Presentations

darrenrogan

aws s3api delete-objects --bucket bucket-name --delete "$(aws s3api list-object-versions --bucket "bucket_name" --output=json --query='{Objects: DeleteMarkers[].{Key:Key,VersionId:VersionId}}')"

nathanmalishev

Version: 2012-10-17
Statement:
  - Effect: Allow
    Action:
      - 'cloudformation:CreateChangeSet'
      - 'cloudformation:DescribeChangeSet'
      - 'cloudformation:ExecuteChangeSet'
      - 'cloudformation:DescribeStacks'
    Resource:
      - 'arn:aws:cloudformation:<region>:<account_no>:stack/<roles_permission_stack_name>/*'

bwhaley

# Quick script to configure routes for a VPC peering connection
# Searches a region for all peering connection and prompts to choose one
# Configures routes between the peered networks for all routing tables
# STS/AssumeRole not implemented cross-account peering. Instead, 
# Choose accepter/requestor depending on which credentials are set in the environment
# Enter either IPv4 and IPv6 route destinations
# Example usage:
#  ( Assuming boto credentials are configured)
#   $ pip install boto3
#   $ python3.6 peer_routes.py

ivawzh

Resources:

  # ============================ Application Load Balancer ============================

  LoadBalancer:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Ref AWS::StackName
      Scheme: internet-facing
      Subnets: !Split

nmarley

import boto3
import base64

if __name__ == '__main__':
    session = boto3.session.Session()

    kms = session.client('kms')

    encrypted_password = 'AQECAHjgTiiE7TYRGp5Irf8jQ3HzlaQaHGYgsUJDaavnHcFm0gAAAGswaQYJKoZIhvcNAQcGoFwwWgIBADBVBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDwxVQuG0oVwpkU7nQIBEIAoVGk1/wpserb+GVUOzE7PiL/Nr9fTDFKZfpKpF0ip2ct4B2q0Wn6ZZw=='
    binary_data = base64.b64decode(encrypted_password)

chris-smith-zocdoc

import os

import boto.utils
import boto3
import requests
import datetime
import time


def get_contents(filename):

vasanthk

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?