filimonov

# uname -a
# Linux ip-172-31-1-128 5.4.0-1038-aws #40-Ubuntu SMP Fri Feb 5 23:53:34 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux

git clone --depth=1 --branch=docker_server_from_ci_builds https://github.com/filimonov/ClickHouse.git 

cd ClickHouse/docker/server/
docker build . --network host --build-arg single_binary_location="https://builds.clickhouse.tech/master/aarch64/clickhouse"
docker image ls
docker tag 98f169cda25a altinity/clickhouse-server:21.4.1.6307-testing-arm
docker login

filimonov

User-level diff between 20.4.4.18 and 20.12.7.3

• Number of PR 2167
• Number of commits 9179
• Number of authors 287

Tables in system database

| name | groupArray(version) |

filimonov

How to add test queries to ClickHouse CI

ClickHouse has hundreds (or even thousands) of features. Every commit get checked by a complex set of tests containing many thousands of test cases.

The core functionality is very well tested, but some corner-cases and different combinations of features can be uncovered with ClickHouse CI.

Most of the bugs/regressions we see happen in that 'grey area' where test coverage is poor.

And we are very interested in covering most of the possible scenarios and feature combinations used in real life by tests.

filimonov

Africa/Juba	(73475999,'1972-04-30 12:59:59','1972-04-30 11:59:59')
Africa/Juba	(73476000,'1972-04-30 13:00:00','1972-04-30 12:00:00')
Africa/Juba	(73476001,'1972-04-30 13:00:01','1972-04-30 12:00:01')
Africa/Juba	(325666799,'1980-04-27 09:59:59','1980-04-27 08:59:59')
Africa/Juba	(325666800,'1980-04-27 10:00:00','1980-04-27 09:00:00')
Africa/Juba	(325666801,'1980-04-27 10:00:01','1980-04-27 09:00:01')
Africa/Juba	(388566009,'1982-04-25 10:00:09','1982-04-25 09:00:09')
Africa/Juba	(388566010,'1982-04-25 10:00:10','1982-04-25 09:00:10')
Africa/Juba	(388566011,'1982-04-25 10:00:11','1982-04-25 09:00:11')
Africa/Khartoum	(73475999,'1972-04-30 12:59:59','1972-04-30 11:59:59')

filimonov

#!/bin/bash
set -ex
set -o pipefail
trap "exit" INT TERM
trap "kill $(jobs -pr) ||:" EXIT

stage=${stage:-}
script_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

rm -rf data {affected-queries,shm-traces,trace-filtered,flamegraph-query-runs}.tsv {interesting-addrs-,stacks.}{left,right}.{tsv,rep} ||:

filimonov

#!/bin/bash
set -e

rm -rf {cumulative,histograms,stats,metric-names}.{tsv,png} histogram-*.png ||:

clickhouse-local --query "
-- leave only statistically significant changes of metrics of sufficient magnitude
create view metric_stats_filtered as
    select *
    from file('report/query-metric-stats.tsv', TSVWithNamesAndTypes,

filimonov

1. File->Preferences->Settings terminal.integrated.scrollback, Increase the buffer size for terminal (i use 100000)

2. File->Prefernces->Keyboard shortcuts workbench.action.terminal.runSelectedText set up some binding like Ctrl+Shift+'

3. Open terminal window: Terminal->New Terminal

4. Run clickhouse-client in a terminal.

filimonov

# docker build -t yandex/clickhouse-server-with-odbc .
# docker start --name ch-odbc-roundtrip -d yandex/clickhouse-server-with-odbc
# docker exec -it ch-odbc-roundtrip clickhouse-client -mn

FROM yandex/clickhouse-server

ARG odbc_driver_url="https://github.com/ClickHouse/clickhouse-odbc/releases/download/v1.1.3.20200115/clickhouse-odbc-1.1.3-Linux.tar.gz"

RUN apt-get update \
    && apt-get install --yes --no-install-recommends unixodbc \

filimonov

Goofys

https://github.com/kahing/goofys

apt-get install fuse

wget https://bit.ly/goofys-latest
chmod +x goofys-latest
cp goofys-latest /usr/local/bin/goofys

filimonov

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

1. List the topics to which the group is subscribed

kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

2. Reset the consumer offset for a topic (preview)