Skip to content

Instantly share code, notes, and snippets.

@jrwest
Last active December 15, 2015 16:09
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jrwest/588d85bbeaebaf940e59 to your computer and use it in GitHub Desktop.
Save jrwest/588d85bbeaebaf940e59 to your computer and use it in GitHub Desktop.

Example 1

Existing riak-admin transfers usage, single call. Bitcask backend.

$ dev/dev1/bin/riak-admin transfers                                                                                            'dev2@127.0.0.1' waiting to handoff 20 partitions
'dev1@127.0.0.1' waiting to handoff 6 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 570899077082383952423314387779798054553098649600
started: 2013-04-01 18:46:44 [19.07 s ago]
last update: 2013-04-01 18:47:02 [808.11 ms ago]
total size: 224 objects
objects transferred: 180

                           10 Objs/s
  dev1@127.0.0.1 |==================     |  80%   dev2@127.0.0.1
                           1.96 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 479555224749202520035584085735030365824602865664
started: 2013-04-01 18:46:50 [13.46 s ago]
last update: 2013-04-01 18:47:02 [1.29 s ago]
total size: 247 objects
objects transferred: 120

                           10 Objs/s
  dev1@127.0.0.1 |===========            |  48%   dev2@127.0.0.1
                           1.96 KB/s

Example 2

call riak-admin transfers every 2 seconds, max 5 times bitcask backend

$ dev/dev1/bin/riak-admin transfers 2 5                                                                                        
'dev2@127.0.0.1' waiting to handoff 60 partitions
'dev1@127.0.0.1' waiting to handoff 4 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 319703483166135013357056057156686910549735243776
started: 2013-04-01 18:45:40 [7.43 s ago]
last update: 2013-04-01 18:45:46 [1.35 s ago]
total size: 85 objects
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |================       |  70%   dev2@127.0.0.1
                           1.96 KB/s


'dev2@127.0.0.1' waiting to handoff 60 partitions
'dev1@127.0.0.1' waiting to handoff 4 partitions

Active Transfers:


'dev2@127.0.0.1' waiting to handoff 58 partitions
'dev1@127.0.0.1' waiting to handoff 6 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 502391187832497878132516661246222288006726811648
started: 2013-04-01 18:45:50 [1.45 s ago]
last update: no updates seen
total size: unknown
objects transferred: unknown

                            unknown
  dev1@127.0.0.1 |                       |   0%   dev2@127.0.0.1
                            unknown

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 205523667749658222872393179600727299639115513856
started: 2013-04-01 18:45:50 [1.45 s ago]
last update: no updates seen
total size: unknown
objects transferred: unknown

                            unknown
  dev1@127.0.0.1 |                       |   0%   dev2@127.0.0.1
                            unknown


'dev2@127.0.0.1' waiting to handoff 58 partitions
'dev1@127.0.0.1' waiting to handoff 6 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 502391187832497878132516661246222288006726811648
started: 2013-04-01 18:45:50 [3.46 s ago]
last update: 2013-04-01 18:45:52 [1.43 s ago]
total size: 126 objects
objects transferred: 20

                           10 Objs/s
  dev1@127.0.0.1 |===                    |  15%   dev2@127.0.0.1
                           1.95 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 205523667749658222872393179600727299639115513856
started: 2013-04-01 18:45:50 [3.46 s ago]
last update: 2013-04-01 18:45:52 [1.43 s ago]
total size: 130 objects
objects transferred: 20

                           10 Objs/s
  dev1@127.0.0.1 |===                    |  15%   dev2@127.0.0.1
                           1.95 KB/s


'dev2@127.0.0.1' waiting to handoff 58 partitions
'dev1@127.0.0.1' waiting to handoff 6 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 502391187832497878132516661246222288006726811648
started: 2013-04-01 18:45:50 [5.47 s ago]
last update: 2013-04-01 18:45:54 [1.42 s ago]
total size: 126 objects
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |=======                |  31%   dev2@127.0.0.1
                           1.95 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 205523667749658222872393179600727299639115513856
started: 2013-04-01 18:45:50 [5.47 s ago]
last update: 2013-04-01 18:45:54 [1.42 s ago]
total size: 130 objects
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |=======                |  30%   dev2@127.0.0.1
                           1.96 KB/s

Example 3:

call riak-admin transfers every 2 seconds until CTRL+C is pressed or nodetool timeout reached. memory backend, size is dynamic (fold is not a snapshot) reflected in the output.

$ dev/dev1/bin/riak-admin transfers 2                                                                                          {11:54}
'dev2@127.0.0.1' waiting to handoff 60 partitions
'dev1@127.0.0.1' waiting to handoff 4 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 137015778499772148581595453067151533092743675904
started: 2013-04-01 18:54:23 [4.40 s ago]
last update: 2013-04-01 18:54:27 [351.56 ms ago]
total size: 83 objects
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |===========            |  48%   dev2@127.0.0.1
                           1.97 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 228359630832953580969325755111919221821239459840
started: 2013-04-01 18:54:23 [4.40 s ago]
last update: 2013-04-01 18:54:27 [355.84 ms ago]
total size: 96 objects
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |=========              |  41%   dev2@127.0.0.1
                           1.97 KB/s


'dev2@127.0.0.1' waiting to handoff 60 partitions
'dev1@127.0.0.1' waiting to handoff 4 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 137015778499772148581595453067151533092743675904
started: 2013-04-01 18:54:23 [6.41 s ago]
last update: 2013-04-01 18:54:29 [340.60 ms ago]
total size: 86 objects
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |================       |  69%   dev2@127.0.0.1
                           1.96 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 228359630832953580969325755111919221821239459840
started: 2013-04-01 18:54:23 [6.41 s ago]
last update: 2013-04-01 18:54:29 [344.06 ms ago]
total size: 104 objects
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |=============          |  57%   dev2@127.0.0.1
                           1.97 KB/s


'dev2@127.0.0.1' waiting to handoff 60 partitions
'dev1@127.0.0.1' waiting to handoff 4 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 228359630832953580969325755111919221821239459840
started: 2013-04-01 18:54:23 [8.42 s ago]
last update: 2013-04-01 18:54:31 [331.44 ms ago]
total size: 115 objects
objects transferred: 80

                           10 Objs/s
  dev1@127.0.0.1 |================       |  69%   dev2@127.0.0.1
                           1.97 KB/s


'dev2@127.0.0.1' waiting to handoff 59 partitions
'dev1@127.0.0.1' waiting to handoff 5 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 228359630832953580969325755111919221821239459840
started: 2013-04-01 18:54:23 [10.43 s ago]
last update: 2013-04-01 18:54:33 [317.76 ms ago]
total size: 123 objects
objects transferred: 100

                           10 Objs/s
  dev1@127.0.0.1 |==================     |  81%   dev2@127.0.0.1
                           1.97 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 296867520082839655260123481645494988367611297792
started: 2013-04-01 18:54:33 [434.18 ms ago]
last update: no updates seen
total size: unknown
objects transferred: unknown

                            unknown
  dev1@127.0.0.1 |                       |   0%   dev2@127.0.0.1
                            unknown


'dev2@127.0.0.1' waiting to handoff 59 partitions
'dev1@127.0.0.1' waiting to handoff 5 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 296867520082839655260123481645494988367611297792
started: 2013-04-01 18:54:33 [2.44 s ago]
last update: 2013-04-01 18:54:35 [420.73 ms ago]
total size: 101 objects
objects transferred: 20

                           10 Objs/s
  dev1@127.0.0.1 |====                   |  19%   dev2@127.0.0.1
                           1.97 KB/s


'dev2@127.0.0.1' waiting to handoff 59 partitions
'dev1@127.0.0.1' waiting to handoff 5 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 296867520082839655260123481645494988367611297792
started: 2013-04-01 18:54:33 [4.45 s ago]
last update: 2013-04-01 18:54:37 [409.50 ms ago]
total size: 109 objects
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |========               |  36%   dev2@127.0.0.1
                           1.97 KB/s


'dev2@127.0.0.1' waiting to handoff 59 partitions
'dev1@127.0.0.1' waiting to handoff 5 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 296867520082839655260123481645494988367611297792
started: 2013-04-01 18:54:33 [6.46 s ago]
last update: 2013-04-01 18:54:39 [395.91 ms ago]
total size: 116 objects
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |===========            |  51%   dev2@127.0.0.1
                           1.97 KB/s


^C

Example 4

eleveldb backend w/ [1]. uses bytes instead of number of keys as upper bound. NOTE: The current changes in [1] do not include memtable byte size, so if not enough keys are written or the node is not restarted the progress cannot yet be tracked.

[1] basho/leveldb#74

$ dev/dev1/bin/riak-admin transfers 2 20                                                                                       'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:28:00 [2.96 s ago]
last update: 2013-04-01 19:28:02 [893.30 ms ago]
total size: 50921 bytes
objects transferred: 20

                           10 Objs/s
  dev1@127.0.0.1 |=                      |   7%   dev2@127.0.0.1
                           1.92 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:28:00 [2.96 s ago]
last update: 2013-04-01 19:28:02 [896.00 ms ago]
total size: 50468 bytes
objects transferred: 20

                           10 Objs/s
  dev1@127.0.0.1 |=                      |   8%   dev2@127.0.0.1
                           1.92 KB/s


'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:28:00 [4.97 s ago]
last update: 2013-04-01 19:28:05 [879.07 ms ago]
total size: 50921 bytes
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |===                    |  15%   dev2@127.0.0.1
                           1.94 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:28:00 [4.97 s ago]
last update: 2013-04-01 19:28:05 [881.53 ms ago]
total size: 50468 bytes
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |===                    |  16%   dev2@127.0.0.1
                           1.94 KB/s


'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:28:00 [6.98 s ago]
last update: 2013-04-01 19:28:07 [863.13 ms ago]
total size: 50921 bytes
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |=====                  |  23%   dev2@127.0.0.1
                           1.94 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:28:00 [6.98 s ago]
last update: 2013-04-01 19:28:07 [867.58 ms ago]
total size: 50468 bytes
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |=====                  |  24%   dev2@127.0.0.1
                           1.94 KB/s


'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:28:00 [8.99 s ago]
last update: 2013-04-01 19:28:09 [848.60 ms ago]
total size: 50921 bytes
objects transferred: 80

                           10 Objs/s
  dev1@127.0.0.1 |=======                |  31%   dev2@127.0.0.1
                           1.95 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:28:00 [9.00 s ago]
last update: 2013-04-01 19:28:09 [851.34 ms ago]
total size: 50468 bytes
objects transferred: 80

                           10 Objs/s
  dev1@127.0.0.1 |=======                |  32%   dev2@127.0.0.1
                           1.95 KB/s


'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:28:00 [11.00 s ago]
last update: 2013-04-01 19:28:11 [827.57 ms ago]
total size: 50921 bytes
objects transferred: 100

                           10 Objs/s
  dev1@127.0.0.1 |=========              |  39%   dev2@127.0.0.1
                           1.95 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:28:00 [11.01 s ago]
last update: 2013-04-01 19:28:11 [829.42 ms ago]
total size: 50468 bytes
objects transferred: 100

                           10 Objs/s
  dev1@127.0.0.1 |=========              |  40%   dev2@127.0.0.1
                           1.95 KB/s

Example 5

When backend does not support datasize estimation (multibackend or eleveldb w/o [1]):

~/sandbox/basho/ongoing/handoff-progress/riak:jrw-handoff-progress ✗ $ dev/dev1/bin/riak-admin transfers 2 20                                                                                       {12:39}'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:39:53 [4.18 s ago]
last update: 2013-04-01 19:39:57 [74.19 ms ago]
total size: unknown
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |=======================| N/A%   dev2@127.0.0.1
                           1.94 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:39:53 [4.18 s ago]
last update: 2013-04-01 19:39:57 [76.31 ms ago]
total size: unknown
objects transferred: 40

                           10 Objs/s
  dev1@127.0.0.1 |=======================| N/A%   dev2@127.0.0.1
                           1.94 KB/s


'dev2@127.0.0.1' waiting to handoff 64 partitions

Active Transfers:

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 22835963083295358096932575511191922182123945984
started: 2013-04-01 19:39:53 [6.19 s ago]
last update: 2013-04-01 19:39:59 [55.01 ms ago]
total size: unknown
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |=======================| N/A%   dev2@127.0.0.1
                           1.95 KB/s

transfer type: ownership_handoff
vnode type: riak_kv_vnode
partition: 45671926166590716193865151022383844364247891968
started: 2013-04-01 19:39:53 [6.20 s ago]
last update: 2013-04-01 19:39:59 [57.22 ms ago]
total size: unknown
objects transferred: 60

                           10 Objs/s
  dev1@127.0.0.1 |=======================| N/A%   dev2@127.0.0.1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment