Skip to content

Instantly share code, notes, and snippets.

@tom-clickhouse
Created October 17, 2023 19:32
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tom-clickhouse/3570f02b38e8b5f11351cd2d446f7d8c to your computer and use it in GitHub Desktop.
Save tom-clickhouse/3570f02b38e8b5f11351cd2d446f7d8c to your computer and use it in GitHub Desktop.
min_insert_block_size_rows = 80_000_000
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- 8 insert threads
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- clickhouse-client output --------------------------------------------------------------------------------------------
0 rows in set. Elapsed: 10824.317 sec. Processed 65.33 billion rows, 9.60 TB (6.04 million rows/s., 887.18 MB/s.)
Peak memory usage: 227.94 GiB.
-- number of active parts at ingest finish -----------------------------------------------------------------------------
┌─parts─┐
│ 467 │
└───────┘
-- time to less than 3k active parts ----------------------------------------------------------------------------------
┌─seconds─┐
│ 10824 │
└─────────┘
-- number of initial parts written to storage --------------------------------------------------------------------------
┌─parts─────────┬─rows_max──────┬─rows_avg──────┬─rows_min─────┬─size_uncompressed_avg─┬─size_compressed_avg─┬─rows_total────┐
│ 1.64 thousand │ 40.01 million │ 39.76 million │ 2.10 million │ 8.96 GiB │ 768.94 MiB │ 65.33 billion │
└───────────────┴───────────────┴───────────────┴──────────────┴───────────────────────┴─────────────────────┴───────────────┘
-- number of initial parts written to storage per cluster node ---------------------------------------------------------
┌─n─┬─parts──┬─rows_total────┐
│ 1 │ 520.00 │ 20.66 billion │
│ 2 │ 616.00 │ 24.41 billion │
│ 3 │ 507.00 │ 20.27 billion │
└───┴────────┴───────────────┘
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- 4 insert threads
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- clickhouse-client output --------------------------------------------------------------------------------------------
0 rows in set. Elapsed: 18853.020 sec. Processed 65.33 billion rows, 9.60 TB (3.47 million rows/s., 509.37 MB/s.)
Peak memory usage: 227.52 GiB.
-- number of active parts at ingest finish -----------------------------------------------------------------------------
┌─parts─┐
│ 154 │
└───────┘
-- time to less than 3k active parts ----------------------------------------------------------------------------------
┌─seconds─┐
│ 18853 │
└─────────┘
-- number of initial parts written to storage --------------------------------------------------------------------------
┌─parts──┬─rows_max──────┬─rows_avg──────┬─rows_min──────┬─size_uncompressed_avg─┬─size_compressed_avg─┬─rows_total────┐
│ 824.00 │ 80.01 million │ 79.29 million │ 14.10 million │ 17.91 GiB │ 1.45 GiB │ 65.33 billion │
└────────┴───────────────┴───────────────┴───────────────┴───────────────────────┴─────────────────────┴───────────────┘
-- number of initial parts written to storage per cluster node ---------------------------------------------------------
┌─n─┬─parts──┬─rows_total────┐
│ 1 │ 260.00 │ 20.55 billion │
│ 2 │ 252.00 │ 20.05 billion │
│ 3 │ 312.00 │ 24.73 billion │
└───┴────────┴───────────────┘
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment