kvoli

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,

kvoli

 store rebalancer
 phase(1)
 while qps > max threshold and a target hot range exists
   transfer lease to replica with lowest qps for hot range

 (phase 2)

 while qps > max threshold
   get new replica set, using the allocator add target and remove target N times
   call relocate range and send lease to store with lowest qps

kvoli

standard

can query w/ grafana

workload produced artifacts and total value is ops / duration (all operation types and all durations added).

• ycsb
• kv
• ledger

kvoli

nightly benchmark testing sketch

currently we have roachperf which is not well suited to arbitrary collection.
The index is stored in a javascript file and files on disk for details.

We should instead have a simple library that allows for exporting to a better
visualization platform. 

- on adhoc roachprod clusters, add --scrape to export prom stats to influxdb
- on adhoc roachprod clusters, add --scrape-ttl to set scrape export ttl (important for short lived rp clusters)

kvoli

follower-to-leaseholder request statistics

problem

The leaseholder replica currently has no information on the number of reads that occur on a follower replica, aside from the follower read request contribution to the follower's store QPS. The leaseholder is the decision maker for the replica placement for the range.

The writes that the leaseholder observes should be symmetric to the writes that

kvoli

0 runs so far, 0 failures, over 5s
i



0 runs so far, 0 failures, over 10s
16 runs so far, 0 failures, over 15s
16 runs so far, 0 failures, over 20s
32 runs so far, 0 failures, over 25s
32 runs so far, 0 failures, over 30s

kvoli

import random

def check(candidates, watermark):
    mean = sum(candidates) / len(candidates)
    targets, excluded = [], []
    for c in candidates:
        if c > 20 and c > mean * watermark:
            excluded.append(c)
            continue
        targets.append(c)

kvoli

import yaml
import sys

FILENAME = "prometheus.yml"

def format_file(filename, scrape_endpoints):
    contents = {}
    with open(filename) as f:
        contents = yaml.load(f, Loader=yaml.FullLoader)
        for scrape_config in contents["scrape_configs"]:

kvoli

• Edsger W. Dijkstra for “Self-stabilizing systems in spite of distributed control” in Communications of the ACM, 17(11):643-644, November 1974.
• Leslie Lamport for “Time, Clocks, and the Ordering of Events in a Distributed System” in Communications of the ACM, 21(7):558-565, July 1978.
• Marshall Pease, Robert Shostak, and Leslie Lamport for “Reaching agreement in the presence of faults” in Journal of the ACM, 27(1):228-234, April 1980.
• R. G. Gallager, P. A. Humblet, and P. M. Spira for “A Distributed Algorithm for Minimum-Weight Spanning Trees” in ACM Transactions on Programming Languages and Systems, 5(1):66-77, January 1983.
• Michael Ben-Or for “Another Advantage of Free Choice: Completely Asynchronous Agreement Protocols” in Proceedings of the Second ACM Symposium on Principles of Distributed Computing, pages 27-30, August 1983.
• Michael O. Rabin for “Randomized Byzantine Generals” in Proceedings of Twenty-Fourth IEEE Annual Symposium on Foundations of Computer Science, pages

kvoli

# This is the main configuration file for Spigot.
# As you can see, there's tons to configure. Some options may impact gameplay, so use
# with caution, and make sure you know what each option does before configuring.
# For a reference for any variable inside this file, check out the Spigot wiki at
# http://www.spigotmc.org/wiki/spigot-configuration/
# 
# If you need help with the configuration or have any questions related to Spigot,
# join us at the IRC or drop by our forums and leave a post.
# 
# IRC: #spigot @ irc.spi.gt ( http://www.spigotmc.org/pages/irc/ )