Skip to content

Instantly share code, notes, and snippets.

View filimonov's full-sized avatar

filimonov filimonov

View GitHub Profile
@StevenACoffman
StevenACoffman / Docker Best Practices.md
Last active April 29, 2024 08:36
Docker Best Practices

Mistakes to Avoid: Docker Antipatterns

Whichever route you take to implementing containers, you’ll want to steer clear of common pitfalls that can undermine the efficiency of your Docker stack.

Don’t run too many processes inside a single container

The beauty of containers—and an advantage of containers over virtual machines—is that it is easy to make multiple containers interact with one another in order to compose a complete application. There is no need to run a full application inside a single container. Instead, break your application down as much as possible into discrete services, and distribute services across multiple containers. This maximizes flexibility and reliability.

Don’t install operating systems inside Docker containers

It is possible to install a complete Linux operating system inside a container. In most cases, however, this is not necessary. If your goal is to host just a single application or part of an application in the container, you need to install only the essential

Byobu is a suite of enhancements to tmux, as a command line
tool providing live system status, dynamic window management,
and some convenient keybindings:
F1 * Used by X11 *
Shift-F1 Display this help
F2 Create a new window
Shift-F2 Create a horizontal split
Ctrl-F2 Create a vertical split
Ctrl-Shift-F2 Create a new session
KEYBINDINGS
byobu keybindings can be user defined in /usr/share/byobu/keybindings/ (or within .screenrc if byobu-export was used). The common key bindings
are:
F2 - Create a new window
F3 - Move to previous window
F4 - Move to next window
@Slach
Slach / Dockerfile
Last active February 6, 2024 23:20
clickhouse with odbc, for docker-compose and kubernetes
FROM yandex/clickhouse-server:${CLICKHOUSE_VERSION:-latest}
USER root
ARG UBUNTU_NAME=focal
ARG UBUNTU_VERSION=20.04
RUN echo "Begin ODBC install" && \
#MySQL repo
wget -qO- "http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xa4a9406876fcbd3c456770c88c718d3b5072e1f5" | apt-key add - && \
echo "deb http://repo.mysql.com/apt/ubuntu/ ${UBUNTU_NAME} mysql-8.0" >/etc/apt/sources.list.d/mysql-oracle.list && \
@den-crane
den-crane / uniqState_uniqMerge
Last active December 29, 2023 02:02
CH AggregatingMergeTree uniqState uniqMerge
drop table z;
drop table mvz;
create table z(d Date, u String) Engine=MergeTree partition by tuple() order by tuple();
CREATE MATERIALIZED VIEW mvz ENGINE = AggregatingMergeTree(d, (d), 8192) as select d, uniqState(u) as us from z group by d
insert into z select today()-number%571, concat('usr',toString(rand()%664579)) from numbers(100000000);
optimize table mvz final;
optimize table z final;
select (uniqMerge(us)) as unique from mvz group by d order by d;
571 rows in set. Elapsed: 0.300 sec.
@den-crane
den-crane / CH-event-sourcing-using-AggregatingMergeTree
Last active December 29, 2023 02:02
event sourcing using AggregatingMergeTree
DROP TABLE IF EXISTS states_raw;
DROP TABLE IF EXISTS final_states_by_day;
DROP TABLE IF EXISTS final_states_by_day_mv;
CREATE TABLE states_raw
(
process String,
state String,
stateint Int64,
statevalue Float64,
@den-crane
den-crane / Clickhouse-fast-not-exists
Last active November 3, 2023 10:53
Clickhouse fast not exists
ClickHouse server version 18.14.12 revision 54409.
create table data(K Int64, V String) engine=MergeTree order by K;
insert into data select number, toString(number) from numbers(100,100000000);
optimize table data final;
create table buffer(K Int64, V String) engine=Memory;
insert into buffer select number, toString(number) from numbers(0,1000);
@den-crane
den-crane / gist:6eff375752a236a456e1b3dc2ca7db62
Last active September 30, 2023 08:43
Clickhouse example AggregatingMergeTree, (max, min, avg ) State / Merge
DROP TABLE IF EXISTS requests;
CREATE TABLE requests (
request_date Date,
request_time DateTime,
response_time Int,
request_uri String)
ENGINE = MergeTree(request_date, (request_time, request_uri), 8192);
@den-crane
den-crane / AggregatingMergeTree-event-enrichment
Last active September 7, 2023 17:32
AggregatingMergeTree-event-enrichment
CREATE TABLE states_raw(
d date,
uid UInt64,
first_name String,
last_name String,
modification_timestamp_mcs DateTime64(3) default now64(3)
) ENGINE = Null;
CREATE TABLE final_states_by_month(
d date,

Building Kafka from the Hardware - up

  • Higher Message Retention ? - Increase disk size
  • Higher Message Throughput ? - Increase network capacity
  • Higher Producer Performance ? - Increase Disk I/O speed
  • Higher Consumer Performance ? - Increase Memory

Critical Configurations (Consumer)

  • queued.min.messages
  • fetch.wait.max.ms
  • socket.blocking.max.ms