InKolev

Building Kafka from the Hardware - up

• Higher Message Retention ? - Increase disk size
• Higher Message Throughput ? - Increase network capacity
• Higher Producer Performance ? - Increase Disk I/O speed
• Higher Consumer Performance ? - Increase Memory

Critical Configurations (Consumer)

• queued.min.messages
• fetch.wait.max.ms
• socket.blocking.max.ms

InKolev

Multi-tenant systems

Check if the observed behavior is the same for other tenants, not just for a single one. The wider the dataset, the easier it becomes to isolate the probem.

Check for data bleed accross tenants

• Configuration mix
• HTTP requests routed to the wrong tenant
• Message bus listeners consuming and processing data for the wrong tenant or for more than 1 tenants.

InKolev

Find largest files on disk (where size is bigger than 100MB)

sudo find / -xdev -type f -size +100M -exec du -sh {} ';' | sort -rh | head -n50

PM2 node process manager

pm2 start ecosystem.config.js

Sample ecosystem file

InKolev

Memory Management in .NET

"If you don't ultimately have a root reference, then you can't be accessed, so you must die."

Brief overview

SOH:

• Allocates objects smaller than 85K
• Objects are allocated on top of each other, a Next Object Pointer (NOP) represents the location where the next object will be allocated.
• Because the NOP position is known, a new object can be immediately allocated without the overhead of looking for space in a Free Space tables.

InKolev

Load/Stress testing

Netling

Netling is a load tester client for easy web testing. It is extremely fast while using little CPU or memory.

WebSurge

Quick and easy URL and load testing for your Web applications.
West Wind WebSurge is a Windows application that helps you to easily capture and play back URL requests, either individually for easy application, service and API testing scenarios, or under heavy load for stress testing scenarios. WebSurge makes it easy to save, share and reuse URL sessions you have created, so you can easily start up sessions again on your own or somebody else's machine.

InKolev

Conway's law

"Any organization that designs a system, will produce a design
whose structure is a copy of the organization's communication structure."

Jimmy's law

"A broken, dysfunctional organization, driven by meeting unhealthy goals and metrics
will produce broken, dysfunctional systems, even if the developers in that organization are good."

Kosh (Babylon 5)

"The avalanche has already started. It is too late for the pebbles to vote."

InKolev

Bet Settlement Worker (x1)

• 40-50k items per minute (3000 cursor size, 1000 concurrency counter, 500 batch size, 250 ms timeout)

Account Operation Workers (x6)

• 25-45k items per minute (with 3ms SQL Delay)
• 30-50k items per minute (with 0ms SQL Delay)

Combo Bonus Workers (x6) (Slowest)

• 18-40k items per minute (with 3ms SQL Delay)
• 25-45k items per minute (with 0ms SQL Delay

InKolev

1. Get authorization token POST
   http://craftcluster/api/FacebookLogin/GetAuthorizationToken?facebookAccessToken=...&facebookProfileId=...

InKolev

Local IP addresses

Kafka: 172.18.0.4
Zookeeper: 172.18.0.2

Useful commands

• Create a topic named integration-test-topic with two partitions:

  • kafka-topics --create --zookeeper 172.18.0.2:3001 --replication-factor 1 --partitions 2 --topic integration-test-topic

• Check topic information:

• kafka-topics --zookeeper 172.18.0.2:3001 --describe --topic integration-test-topic

InKolev

Unstage all files staged for commit

• Command

git reset HEAD -- .

Revert to previous commit, discarding all changes made locally

• Command

git reset --hard HEAD