mks0ff

/*
 *  This query will run across an entire organization looking at tables across every project
 *  and shows how they will compare on compressed and uncompressed storage.
 *
 *  Region Notes:
 *  This query will only read from a single region or multi-region at a time. It's
 *  currently not possible to read this data from across all
 *  
 *  By default this reads from the US multi-region, so this might need to be changed if
 *  your data lives elsewhere.

mks0ff

BigQuery tricks

Data

Click to see script

with
dta as (
select '+++bHyZZlp6KRHJBj5Cw8wd48QA=' Id, timestamp('2019-11-07 06:53:39.632 UTC') Ts union all

mks0ff

Default

Powerlevel10k

mks0ff

I've been working with Kafka for over 7 years. I inevitably find myself doing the same set of activities while I'm developing or working with someone else's system. Here's a set of Kafka productivity hacks for doing a few things way faster than you're probably doing them now. 🔥

• Show me all my Kafka topics and their partitions, replicas, and consumers
• Show me the contents of a topic
• Create a Kafka topic
• Produce messages to a Kafka topic
• Validate the schema of messages before producing to a topic
• Do all of this at a distance

Get the tools

mks0ff

Create a schema based on the default one

https://github.com/basho/yokozuna/blob/develop/priv/default_schema.xml

Upload your schema

curl -XPUT -H 'Content-Type: application/xml'  http://localhost:8098/search/schema/drew_schema --data-binary @"my_schema.xml"

mks0ff

Docker - How to cleanup (unused) resources

Cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm

mks0ff

# Pre-req: gcc, gcc-c++, git (to clone), openjdk with debug symbols (centos: java-1.8.0-openjdk-debug.x86_64, ubuntu: openjdk-8-dbg)
# Git clone async-profiler https://github.com/jvm-profiling-tools/async-profiler
# Make sure JAVA_HOME is set and you are logged as same user running the java process
cd async-profiler
make all
# Also git clone FlameGraph in the same directory as you clone async-profiler

# sysctl changes
echo 1 > /proc/sys/kernel/perf_event_paranoid
echo 0 > /proc/sys/kernel/kptr_restrict

mks0ff

apt-get update
apt-get install curl gcc memcached rsync sqlite3 xfsprogs git-core libffi-dev python-setuptools liberasurecode-dev libssl-dev
apt-get install python-coverage python-dev python-nose python-xattr python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip 
apt-get install python-dnspython python-mock
apt-get install xfsprogs rsync
apt-get install swift swift-account swift-container swift-object
apt install python-swiftclient python3-swiftclient
apt-get install emacs

curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/queens

mks0ff

# Add the following 'help' target to your Makefile
# And add help text after each target name starting with '\#\#'
 
help:           ## Show this help.
	@fgrep -h "##" $(MAKEFILE_LIST) | fgrep -v fgrep | sed -e 's/\\$$//' | sed -e 's/##//'
 
# Everything below is an example
 
target00:       ## This message will show up when typing 'make help'
	@echo does nothing

mks0ff

Category theory jargon cheat sheet

A primer/refresher on the category theory concepts that most commonly crop up in conversations about Scala or FP. (Because it's embarassing when I forget this stuff!)

I'll be assuming Scalaz imports in code samples, and some of the code may be pseudo-Scala.

Functor

A functor is something that supports map.