Skip to content

Instantly share code, notes, and snippets.

@tom-clickhouse
Created October 17, 2023 19:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tom-clickhouse/8955d5a57ad4babf84690a10f90da97a to your computer and use it in GitHub Desktop.
Save tom-clickhouse/8955d5a57ad4babf84690a10f90da97a to your computer and use it in GitHub Desktop.
min_insert_block_size_rows = 40_000_000
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- 8 insert threads
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- clickhouse-client output --------------------------------------------------------------------------------------------
0 rows in set. Elapsed: 10824.317 sec. Processed 65.33 billion rows, 9.60 TB (6.04 million rows/s., 887.18 MB/s.)
Peak memory usage: 227.94 GiB.
-- number of active parts at ingest finish -----------------------------------------------------------------------------
┌─parts─┐
│ 467 │
└───────┘
-- time to less than 3k active parts ----------------------------------------------------------------------------------
┌─seconds─┐
│ 10824 │
└─────────┘
-- number of initial parts written to storage --------------------------------------------------------------------------
┌─parts─────────┬─rows_max──────┬─rows_avg──────┬─rows_min─────┬─size_uncompressed_avg─┬─size_compressed_avg─┬─rows_total────┐
│ 1.64 thousand │ 40.01 million │ 39.76 million │ 2.10 million │ 8.96 GiB │ 768.94 MiB │ 65.33 billion │
└───────────────┴───────────────┴───────────────┴──────────────┴───────────────────────┴─────────────────────┴───────────────┘
-- number of initial parts written to storage per cluster node ---------------------------------------------------------
┌─n─┬─parts──┬─rows_total────┐
│ 1 │ 520.00 │ 20.66 billion │
│ 2 │ 616.00 │ 24.41 billion │
│ 3 │ 507.00 │ 20.27 billion │
└───┴────────┴───────────────┘
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- 4 insert threads
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
-- clickhouse-client output --------------------------------------------------------------------------------------------
0 rows in set. Elapsed: 18061.511 sec. Processed 65.33 billion rows, 9.60 TB (3.62 million rows/s., 531.69 MB/s.)
Peak memory usage: 114.23 GiB.
-- number of active parts at ingest finish -----------------------------------------------------------------------------
┌─parts─┐
│ 183 │
└───────┘
-- time to less than 3k active parts ----------------------------------------------------------------------------------
┌─seconds─┐
│ 18061 │
└─────────┘
-- number of initial parts written to storage --------------------------------------------------------------------------
┌─parts─────────┬─rows_max──────┬─rows_avg──────┬─rows_min─────┬─size_uncompressed_avg─┬─size_compressed_avg─┬─rows_total────┐
│ 1.64 thousand │ 40.01 million │ 39.91 million │ 1.27 million │ 9.03 GiB │ 771.49 MiB │ 65.33 billion │
└───────────────┴───────────────┴───────────────┴──────────────┴───────────────────────┴─────────────────────┴───────────────┘
-- number of initial parts written to storage per cluster node ---------------------------------------------------------
┌─n─┬─parts──┬─rows_total────┐
│ 1 │ 497.00 │ 19.84 billion │
│ 2 │ 528.00 │ 21.09 billion │
│ 3 │ 612.00 │ 24.40 billion │
└───┴────────┴───────────────┘
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment