neomantra

High Performance Redis

In response to this brief blog entry, @antirez tweeted for some documentation on high-performance techniques for Redis. What I present here are general high-performance computing (HPC) techniques. The examples are oriented to Redis. but they work well for any program designed to be single- or worker-threaded and asynchronous (e.g. uses epoll).

The motivation for using these techniques is to maximize performance of our system and services. By isolating work, controlling memory, and other tuning, you can achieve significant reduction in latency and increase in throughput.

My perspective comes from the microcosm of my own bare-metal (vs VM), on-premises deployment. It might not be suitable for all scenarios, especially cloud deployments, as I have little experience with HPC there. After some discussion, maybe this can be adapted as [redis.io documentation](https://redis.io/do

antirez

RESP3 specification

Versions history:

• 1.0, 2 May 2018, Initial draft to get community feedbacks.

Background

The Redis protocol has served us well in the past years, showing that, if carefully designed, a simple human readable protocol is not the bottleneck in the client server communication, and that the simplicity of the design is a major advantage in creating a healthy client libraries ecosystem.

Yet the Redis experience has shown that after about six years from its introduction (when it replaced the initial Redis protocol), the current RESP protocol could be improved, especially in order to make client implementations simpler and to support new features.

dstroot

package main

import (
	"io"
	"net/http"
	"sync/atomic"
)

func index() http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

enricofoltran

package main

import (
	"context"
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"

voluntas

### KERNEL TUNING ###

# Increase size of file handles and inode cache
fs.file-max = 2097152

# Do less swapping
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2

angrycub

Job Anti-Affinity in Action

Overview

This guide will walk you through creating and executing a job that will demonstrate Nomad's job anti-affinity rules and, in clusters with memory limited Nomad clients, filtering based resource exhaustion.

Sample Environment

• One Nomad Server Node
• Three Nomad Client Nodes
  • 768 MB RAM total (providing 761 MB RAM in nomad node-status -self)

abraithwaite

package main

import (
	"io/ioutil"
	"log"

	"github.com/naoina/toml"
	"github.com/naoina/toml/ast"
)

antirez

Redis 4.2 roadmap

1. Redis Cluster

• Speed up key -> hashslot association. Now makes RBB loading 4x slower when there are many small keys.
• Better multi data center story
• redis-trib C coded and moved into redis-cli
• Backup / Restore of Cluster
• Non blocking MIGRATE (also consider not using 2X memory)
• Faster resharding
• Bug fixing and stress testing to bring it to next level of maturity

burdandrei

# Templates are updated to be compatible with nomad > 0.7.1
[agent]
  interval = "10s"
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "10s"
  flush_jitter = "3s"
  precision = ""

rondy

FWIW: I (@rondy) am not the creator of the content shared here, which is an excerpt from Edmond Lau's book. I simply copied and pasted it from another location and saved it as a personal note, before it gained popularity on news.ycombinator.com. Unfortunately, I cannot recall the exact origin of the original source, nor was I able to find the author's name, so I am can't provide the appropriate credits.

---

Effective Engineer - Notes

• By Edmond Lau
• Highly Recommended 👍
• http://www.theeffectiveengineer.com/

What's an Effective Engineer?