Skip to content

Instantly share code, notes, and snippets.

@tchaikov
Last active May 8, 2019 15:47
Show Gist options
  • Save tchaikov/6f0e695892c43078d5b6b315e2373860 to your computer and use it in GitHub Desktop.
Save tchaikov/6f0e695892c43078d5b6b315e2373860 to your computer and use it in GitHub Desktop.

build procedure

diff --git a/src/crimson/osd/pg.cc b/src/crimson/osd/pg.cc
index 6cae617f84..50fceece20 100644
--- a/src/crimson/osd/pg.cc
+++ b/src/crimson/osd/pg.cc
@@ -959,6 +959,7 @@ seastar::future<> PG::share_pg_info()
 seastar::future<> PG::wait_for_active()
 {
   logger().debug("wait_for_active: {}", pg_state_string(info.stats.state));
+  return seastar::now();
   if (test_state(PG_STATE_ACTIVE)) {
     return seastar::now();
   } else {
$ git describe
v15.0.0-1094-g6fe8d8223e

$ cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CXX_FLAGS="-fno-omit-frame-pointer" -DWITH_PYTHON3=ON -DWITH_PYTHON2=OFF -DMGR_PYTHON_VERSION=3.7 -DCMAKE_CXX_COMPILER=g++-9 -DCMAKE_C_COMPILER=gcc-9 -DWITH_SYSTEM_BOOST=ON -DWITH_SEASTAR=ON -DWITH_CCACHE=ON -GNinja ..

deployment scenario

 2003  MGR=1 MON=1 OSD=1 MDS=0 RGW=0 ../src/vstart.sh -n -x --without-dashboard --memstore
 2004  bin/ceph osd pool create test-pool 128 128
 2005  bin/ceph osd pool set test-pool size 1
 2006  bin/ceph osd pool set test-pool min_size 1
 2006  bin/init-ceph stop osd.0
 2007  bin/crimson-osd -i 0 -c /var/ssd/ceph/build/ceph.conf --smp 1

classic 4k write

perf stat -p 20725 & bin/rados bench -p test-pool 2 write -b 4096 --no-cleanup ; sleep 1; killall -INT perf_4.19
hints = 1
Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 2 seconds or 0 objects
Object prefix: benchmark_data_gen8_29215
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1      16     24335     24319   94.9912   94.9961 0.000499632 0.000657092
    2       8     48629     48621   94.9557   94.9297 0.000645658 0.000657489
Total time run:         2.00037
Total writes made:      48629
Write size:             4096
Object size:            4096
Bandwidth (MB/sec):     94.961
Stddev Bandwidth:       0.0469563
Max bandwidth (MB/sec): 94.9961
Min bandwidth (MB/sec): 94.9297
Average IOPS:           24310
Stddev IOPS:            12.0208
Max IOPS:               24319
Min IOPS:               24302
Average Latency(s):     0.0006575
Stddev Latency(s):      0.000168804
Max latency(s):         0.0022814
Min latency(s):         0.000215417
 Performance counter stats for process id '27325':
          6,989.85 msec task-clock                #    1.773 CPUs utilized
            82,190      context-switches          # 11759.908 M/sec
             9,185      cpu-migrations            # 1314.208 M/sec
           181,962      page-faults               # 26035.484 M/sec
    22,252,313,579      cycles                    # 3183905.219 GHz
    17,921,165,519      stalled-cycles-frontend   #   80.54% frontend cycles idle
     8,567,805,160      instructions              #    0.39  insn per cycle
                                                  #    2.09  stalled cycles per insn
     1,767,004,431      branches                  # 252826503.219 M/sec
        75,375,927      branch-misses             #    4.27% of all branches
       3.942500285 seconds time elapsed

classic rand read

Total time run:       30.0004
Total reads made:     747872
Read size:            4096
Object size:          4096
Bandwidth (MB/sec):   97.3779
Average IOPS:         24928
Stddev IOPS:          236.816
Max IOPS:             25460
Min IOPS:             24555
Average Latency(s):   0.000636842
Max latency(s):       0.00674459
Min latency(s):       0.000152665
 Performance counter stats for process id '27325':
         67,348.20 msec task-clock                #    2.176 CPUs utilized
         1,136,463      context-switches          # 16874.488 M/sec
            45,069      cpu-migrations            #  669.196 M/sec
             3,544      page-faults               #   52.622 M/sec
   186,375,035,108      cycles                    # 2767343.278 GHz
   149,262,193,011      stalled-cycles-frontend   #   80.09% frontend cycles idle
    72,672,307,542      instructions              #    0.39  insn per cycle
                                                  #    2.05  stalled cycles per insn
    15,067,875,038      branches                  # 223731588.733 M/sec
       591,469,732      branch-misses             #    3.93% of all branches
      30.953713011 seconds time elapsed

crimson 4k write

hints = 1
Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 2 seconds or 0 objects
Object prefix: benchmark_data_gen8_21373
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
    1      16     26674     26658   104.128   104.133 0.000583379 0.000599236
    2      16     49197     49181   96.0503   87.9805 0.000623006  0.00064987
Total time run:         2.00051
Total writes made:      49198
Write size:             4096
Object size:            4096
Bandwidth (MB/sec):     96.0653
Stddev Bandwidth:       11.4214
Max bandwidth (MB/sec): 104.133
Min bandwidth (MB/sec): 87.9805
Average IOPS:           24592
Stddev IOPS:            2923.89
Max IOPS:               26658
Min IOPS:               22523
Average Latency(s):     0.000649877
Stddev Latency(s):      0.00015241
Max latency(s):         0.00192188
Min latency(s):         0.000192272
 Performance counter stats for process id '20725':
          1,545.22 msec task-clock                #    0.413 CPUs utilized
             1,799      context-switches          # 1164.401 M/sec
                 0      cpu-migrations            #    0.000 K/sec
             3,991      page-faults               # 2583.172 M/sec
     5,202,033,368      cycles                    # 3367011.889 GHz
     2,930,386,630      stalled-cycles-frontend   #   56.33% frontend cycles idle
     5,916,514,248      instructions              #    1.14  insn per cycle
                                                  #    0.50  stalled cycles per insn
     1,147,843,421      branches                  # 742940725.566 M/sec
        11,618,358      branch-misses             #    1.01% of all branches
       3.745898340 seconds time elapsed

crimson rand read

Total time run:       30.0004
Total reads made:     908534
Read size:            4096
Object size:          4096
Bandwidth (MB/sec):   118.297
Average IOPS:         30284
Stddev IOPS:          2975.99
Max IOPS:             40737
Min IOPS:             27223
Average Latency(s):   0.000524767
Max latency(s):       0.00388266
Min latency(s):       0.000179931
 Performance counter stats for process id '20725':
         25,604.04 msec task-clock                #    0.828 CPUs utilized
            39,512      context-switches          # 1543.196 M/sec
                 0      cpu-migrations            #    0.000 K/sec
                 0      page-faults               #    0.000 K/sec
    85,556,798,914      cycles                    # 3341540.342 GHz
    44,513,608,676      stalled-cycles-frontend   #   52.03% frontend cycles idle
   111,008,236,355      instructions              #    1.30  insn per cycle
                                                  #    0.40  stalled cycles per insn
    21,644,660,750      branches                  # 845362472.661 M/sec
       128,650,483      branch-misses             #    0.59% of all branches
      30.929741591 seconds time elapsed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment