Skip to content

Instantly share code, notes, and snippets.

@dt

dt/gist:0f658f7a96af08cec26af8f0cfb52d08

Created March 26, 2019 12:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dt/0f658f7a96af08cec26af8f0cfb52d08 to your computer and use it in GitHub Desktop.
Save dt/0f658f7a96af08cec26af8f0cfb52d08 to your computer and use it in GitHub Desktop.
Download ZIP
import timings for tpcc 1k
Raw
gistfile1.txt
distsql import, csv reader (i.e. master): 55m
I190326 04:27:02.291051 1 ccl/workloadccl/cliccl/fixtures.go:325 starting import of 9 tables
W190326 04:27:02.293200 1 ccl/workloadccl/fixture.go:541 error retrieving automatic stats cluster setting: pq: unknown setting: "sql.stats.automatic_collection.enabled"
I190326 04:27:12.273697 84 ccl/workloadccl/fixture.go:516 imported warehouse (10s, 1000 rows, 0 index entries, 53 KiB)
I190326 04:27:15.543774 85 ccl/workloadccl/fixture.go:516 imported district (13s, 10000 rows, 0 index entries, 1005 KiB)
I190326 04:27:16.486648 90 ccl/workloadccl/fixture.go:516 imported item (14s, 100000 rows, 0 index entries, 7.8 MiB)
I190326 04:29:01.189457 89 ccl/workloadccl/fixture.go:516 imported new_order (1m59s, 9000000 rows, 0 index entries, 126 MiB)
I190326 04:43:08.922034 88 ccl/workloadccl/fixture.go:516 imported order (16m7s, 30000000 rows, 60000000 index entries, 1.8 GiB)
I190326 04:46:40.282246 87 ccl/workloadccl/fixture.go:516 imported history (19m38s, 30000000 rows, 60000000 index entries, 3.8 GiB)
I190326 04:52:34.307789 86 ccl/workloadccl/fixture.go:516 imported customer (25m32s, 30000000 rows, 30000000 index entries, 17 GiB)
I190326 05:07:13.915853 91 ccl/workloadccl/fixture.go:516 imported stock (40m12s, 100000000 rows, 100000000 index entries, 31 GiB)
I190326 05:22:03.980945 92 ccl/workloadccl/fixture.go:516 imported order_line (55m2s, 300011520 rows, 300011520 index entries, 23 GiB)
I190326 05:22:04.069794 1 ccl/workloadccl/cliccl/fixtures.go:337 imported 77 GiB bytes in 9 tables (took 55m1.765193416s, 23.91 MiB/s)
I190326 05:22:04.074403 1 ccl/workloadccl/cliccl/fixtures.go:342 fixture is imported; now running consistency checks (ctrl-c to abort)
I190326 05:22:04.205554 1 workload/tpcc/tpcc.go:292 check 3.3.2.1 took 128.364428ms
I190326 05:22:37.803596 1 workload/tpcc/tpcc.go:292 check 3.3.2.2 took 33.597956283s
I190326 05:22:45.963529 1 workload/tpcc/tpcc.go:292 check 3.3.2.3 took 8.159872426s
distsql import, workload reader: 41m
I190326 03:37:57.901772 1 ccl/workloadccl/cliccl/fixtures.go:325 starting import of 9 tables
W190326 03:37:57.903600 1 ccl/workloadccl/fixture.go:541 error retrieving automatic stats cluster setting: pq: unknown setting: "sql.stats.automatic_collection.enabled"
I190326 03:38:14.895806 16 ccl/workloadccl/fixture.go:516 imported warehouse (17s, 1000 rows, 0 index entries, 53 KiB)
I190326 03:38:39.742946 71 ccl/workloadccl/fixture.go:516 imported item (42s, 100000 rows, 0 index entries, 7.8 MiB)
I190326 03:38:43.628684 66 ccl/workloadccl/fixture.go:516 imported district (46s, 10000 rows, 0 index entries, 1005 KiB)
I190326 03:41:22.635784 70 ccl/workloadccl/fixture.go:516 imported new_order (3m25s, 9000000 rows, 0 index entries, 126 MiB)
^[I190326 03:49:55.747766 69 ccl/workloadccl/fixture.go:516 imported order (11m58s, 30000000 rows, 60000000 index entries, 1.8 GiB)
I190326 03:52:25.542637 68 ccl/workloadccl/fixture.go:516 imported history (14m28s, 30000000 rows, 60000000 index entries, 3.8 GiB)
I190326 03:58:04.678145 67 ccl/workloadccl/fixture.go:516 imported customer (20m7s, 30000000 rows, 30000000 index entries, 17 GiB)
I190326 04:10:12.518584 72 ccl/workloadccl/fixture.go:516 imported stock (32m15s, 100000000 rows, 100000000 index entries, 31 GiB)
I190326 04:19:28.650378 73 ccl/workloadccl/fixture.go:516 imported order_line (41m31s, 300011520 rows, 300011520 index entries, 23 GiB)
I190326 04:19:28.749238 1 ccl/workloadccl/cliccl/fixtures.go:337 imported 77 GiB bytes in 9 tables (took 41m30.809992881s, 31.70 MiB/s)
I190326 04:19:28.751902 1 ccl/workloadccl/cliccl/fixtures.go:342 fixture is imported; now running consistency checks (ctrl-c to abort)
I190326 04:19:28.864686 1 workload/tpcc/tpcc.go:292 check 3.3.2.1 took 108.633924ms
I190326 04:19:58.928842 1 workload/tpcc/tpcc.go:292 check 3.3.2.2 took 30.064069655s
I190326 04:20:03.768713 1 workload/tpcc/tpcc.go:292 check 3.3.2.3 took 4.839801552s
I190326 04:24:40.783112 1 workload/tpcc/tpcc.go:292 check 3.3.2.4 took 4m37.014303641s
direct SST import, csv-reader: 67m
I190326 02:14:21.571753 1 ccl/workloadccl/cliccl/fixtures.go:325 starting import of 9 tables
W190326 02:14:21.573355 1 ccl/workloadccl/fixture.go:541 error retrieving automatic stats cluster setting: pq: unknown setting: "sql.stats.automatic_collection.enabled"
I190326 02:14:37.999938 44 ccl/workloadccl/fixture.go:516 imported warehouse (16s, 1000 rows, 0 index entries, 53 KiB)
I190326 02:14:38.966889 45 ccl/workloadccl/fixture.go:516 imported district (17s, 10000 rows, 0 index entries, 1005 KiB)
I190326 02:14:39.711047 66 ccl/workloadccl/fixture.go:516 imported item (18s, 100000 rows, 0 index entries, 7.8 MiB)
I190326 02:15:01.904560 49 ccl/workloadccl/fixture.go:516 imported new_order (40s, 9000000 rows, 0 index entries, 126 MiB)
I190326 02:22:58.994442 48 ccl/workloadccl/fixture.go:516 imported order (8m37s, 30000000 rows, 60000000 index entries, 1.8 GiB)
I190326 02:29:13.246605 46 ccl/workloadccl/fixture.go:516 imported customer (14m52s, 30000000 rows, 30000000 index entries, 17 GiB)
I190326 02:31:08.807293 47 ccl/workloadccl/fixture.go:516 imported history (16m47s, 30000000 rows, 60000000 index entries, 3.8 GiB)
I190326 03:13:40.146521 67 ccl/workloadccl/fixture.go:516 imported stock (59m19s, 100000000 rows, 100000000 index entries, 31 GiB)
I190326 03:21:46.381962 68 ccl/workloadccl/fixture.go:516 imported order_line (1h7m25s, 300011520 rows, 300011520 index entries, 23 GiB)
I190326 03:21:46.514537 1 ccl/workloadccl/cliccl/fixtures.go:337 imported 77 GiB bytes in 9 tables (took 1h7m24.922721896s, 19.52 MiB/s)
I190326 03:21:46.518847 1 ccl/workloadccl/cliccl/fixtures.go:342 fixture is imported; now running consistency checks (ctrl-c to abort)
I190326 03:21:46.769085 1 workload/tpcc/tpcc.go:292 check 3.3.2.1 took 244.097127ms
I190326 03:22:29.076065 1 workload/tpcc/tpcc.go:292 check 3.3.2.2 took 42.306894478s
I190326 03:22:31.793247 1 workload/tpcc/tpcc.go:292 check 3.3.2.3 took 2.717118255s
direct SST import, workload reader: 65m
I190325 23:46:04.012119 1 ccl/workloadccl/cliccl/fixtures.go:325 starting import of 9 tables
W190325 23:46:04.013599 1 ccl/workloadccl/fixture.go:541 error retrieving automatic stats cluster setting: pq: unknown setting: "sql.stats.automatic_collection.enabled"
I190325 23:46:08.527239 16 ccl/workloadccl/fixture.go:516 imported warehouse (5s, 1000 rows, 0 index entries, 53 KiB)
I190325 23:46:17.710403 66 ccl/workloadccl/fixture.go:516 imported district (14s, 10000 rows, 0 index entries, 1006 KiB)
I190325 23:46:17.950734 71 ccl/workloadccl/fixture.go:516 imported item (14s, 100000 rows, 0 index entries, 7.8 MiB)
I190325 23:46:44.821065 70 ccl/workloadccl/fixture.go:516 imported new_order (41s, 9000000 rows, 0 index entries, 126 MiB)
I190325 23:55:04.669523 69 ccl/workloadccl/fixture.go:516 imported order (9m1s, 30000000 rows, 60000000 index entries, 1.8 GiB)
I190325 23:58:25.754681 67 ccl/workloadccl/fixture.go:516 imported customer (12m22s, 30000000 rows, 30000000 index entries, 17 GiB)
I190326 00:03:53.028332 68 ccl/workloadccl/fixture.go:516 imported history (17m49s, 30000000 rows, 60000000 index entries, 3.8 GiB)
I190326 00:14:38.686952 73 ccl/workloadccl/fixture.go:516 imported order_line (28m35s, 300011520 rows, 300011520 index entries, 23 GiB)
I190326 00:51:13.364228 72 ccl/workloadccl/fixture.go:516 imported stock (1h5m9s, 100000000 rows, 100000000 index entries, 31 GiB)
I190326 00:51:13.472110 1 ccl/workloadccl/cliccl/fixtures.go:337 imported 77 GiB bytes in 9 tables (took 1h5m9.443798065s, 20.20 MiB/s)
I190326 00:51:13.473268 1 ccl/workloadccl/cliccl/fixtures.go:342 fixture is imported; now running consistency checks (ctrl-c to abort)
I190326 00:51:13.595508 1 workload/tpcc/tpcc.go:292 check 3.3.2.1 took 118.363187ms
I190326 00:51:54.146610 1 workload/tpcc/tpcc.go:292 check 3.3.2.2 took 40.551008683s
I190326 00:52:04.012232 1 workload/tpcc/tpcc.go:292 check 3.3.2.3 took 9.865544833s
I190326 00:58:24.404512 1 workload/tpcc/tpcc.go:292 check 3.3.2.4 took 6m20.392200215s
I190326 01:05:22.953809 1 workload/tpcc/tpcc.go:292 check 3.3.2.5 took 6m58.549208261s
I190326 01:19:00.555778 1 workload/tpcc/tpcc.go:292 check 3.3.2.7 took 13m37.601884014s
I190326 01:19:41.897256 1 workload/tpcc/tpcc.go:292 check 3.3.2.8 took 41.341400542s
I190326 01:21:03.339123 1 workload/tpcc/tpcc.go:292 check 3.3.2.9 took 1m21.44176641s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment