alexey-milovidov

Old:

Queries executed: 14.

localhost:9000, queries 14, QPS: 0.455, RPS: 1420830167.754, MiB/s: 8107.538, result RPS: 0.455, result MiB/s: 0.000.

0.000%          2.072 sec.
10.000%         2.085 sec.
20.000%         2.112 sec.
30.000%         2.121 sec.

alexey-milovidov

milovidov-desktop :) SELECT abs(uniq(sipHash64(number)) / 10000000 - 1) FROM numbers(10000000)

SELECT abs((uniq(sipHash64(number)) / 10000000) - 1)
FROM numbers(10000000)

Query id: ea428bba-53ff-40c9-848d-5eaa578c52ec

┌─abs(minus(divide(uniq(sipHash64(number)), 10000000), 1))─┐
│                                     0.004632100000000028 │
└──────────────────────────────────────────────────────────┘

alexey-milovidov

milovidov@milovidov-desktop:~/work/ClickHouse$ cat stringstream.cpp
#include <iostream>
#include <sstream>
#include <stdlib.h>
#include <unistd.h>
#include <stdexcept>


void * operator new(std::size_t size)
{

alexey-milovidov

Clang failed to vectorize the following code:

milovidov@milovidov-desktop:~/work/ClickHouse$ cat xors.cpp
#include <cstddef>
#include <vector>

__attribute__((__noinline__))
void xor10arrays(
    const char8_t * arr0,
    const char8_t * arr1,

alexey-milovidov

# -*- coding: utf-8 -*-
import os

from sandbox import sdk2
import logging
import json
import sandbox.common.types.resource as ctr
from sandbox.projects.clickhouse.BaseOnCommitTask.simple_test_task import SimpleDockerBuildTestTask
from sandbox.projects.clickhouse.BaseOnCommitTask.base import NeedToRunDescription
from sandbox.projects.clickhouse.resources import CLICKHOUSE_REPO

alexey-milovidov

milovidov@milovidov-desktop:~/work/sandbox/BaseOnCommitTask$ cat base_text_test_task.py 
# -*- coding: utf-8 -*-
import logging
from sandbox import sdk2
import sandbox.common.types.resource as ctr
import sandbox.common.types.client as ctc

from sandbox.projects.clickhouse.BaseOnCommitTask.base import PostStatuses, NeedToRunDescription
from sandbox.projects.clickhouse.BaseOnCommitTask.test_task import BaseOnCommitTestTask
from sandbox.projects.clickhouse.resources import CLICKHOUSE_REPO_NO_SUBMODULES

alexey-milovidov

/home/milovidov/work/ClickHouse/build> /usr/bin/ninja clickhouse
[1/2] Re-checking globbed directories...
[34/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/net/Net.cpp.o
[35/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/utils/Array.cpp.o
[36/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/utils/crypto/Sha256HMAC.cpp.o
[37/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/platform/linux-shared/Security.cpp.o
[38/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/platform/linux-shared/Time.cpp.o
[39/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/platform/linux-shared/Environment.cpp.o
[40/2051] Building CXX object contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-core/source/utils/base64/Base6

alexey-milovidov

1. Wait-free каталог баз данных в ClickHouse.
Сделано.

2. Оптимизация выполнения GROUP BY, DISTINCT, а также LIMIT BY с учётом сортированности таблицы.
Сделано кроме LIMIT BY.

3. Поддержка использования в ClickHouse систем координации помимо ZooKeeper.
В прототипе, человек может быть захочет доделать.

4. Спекулятивное выполнение распределённых запросов в ClickHouse. Уменьшение количества потоков при распределённых запросах.

alexey-milovidov

^C2020.04.03 04:17:01.224018 [ 22402 ] {} <Trace> BaseDaemon: Received signal 2
2020.04.03 04:17:01.224310 [ 22402 ] {} <Information> Application: Received termination signal (Interrupt)
2020.04.03 04:17:01.224578 [ 22399 ] {} <Trace> ExternalModelsLoader: Configuration of reloadable objects has changed
2020.04.03 04:17:01.224767 [ 22399 ] {} <Trace> ExternalDictionariesLoader: Configuration of reloadable objects has changed
2020.04.03 04:17:01.224885 [ 22399 ] {} <Trace> ExternalDictionariesLoader: Reloadable object 'dim' is removed
2020.04.03 04:17:01.227468 [ 22399 ] {} <Debug> Application: Received termination signal.
2020.04.03 04:17:01.227563 [ 22399 ] {} <Debug> Application: Waiting for current connections to close.
2020.04.03 04:17:02.233853 [ 22399 ] {} <Information> Application: Closed all listening sockets.
2020.04.03 04:17:02.233967 [ 22399 ] {} <Information> Application: Closed connections.
2020.04.03 04:17:02.235802 [ 22399 ] {} <Information> Application: Shutting down storages.

alexey-milovidov

**** Time summary:
Compilation (5345 times):
  Parsing (frontend):         6374.0 s
  Codegen & opts (backend):   5666.7 s

**** Files that took longest to parse (compiler frontend):
 26660 ms: build/contrib/aws-s3-cmake/CMakeFiles/aws_s3.dir/__/aws/aws-cpp-sdk-s3/source/S3Client.cpp.o
 22303 ms: build/dbms/src/Functions/GatherUtils/CMakeFiles/clickhouse_functions_gatherutils.dir/has.cpp.o
 20231 ms: build/contrib/arrow-cmake/CMakeFiles/arrow_static.dir/__/arrow/cpp/src/arrow/compute/kernels/cast.cc.o
 13858 ms: build/dbms/src/Functions/GatherUtils/CMakeFiles/clickhouse_functions_gatherutils.dir/sliceFromRightConstantOffsetBounded.cpp.o