Skip to content

Instantly share code, notes, and snippets.

View antirez's full-sized avatar

Salvatore Sanfilippo antirez

View GitHub Profile
from machine import SPI, Pin
from uc8151 import UC8151
import math, array, time, random
from microfont import MicroFont
import framebuf
spi = SPI(0, baudrate=12000000, phase=0, polarity=0, sck=Pin(18), mosi=Pin(19), miso=Pin(16))
eink = UC8151(spi,cs=17,dc=20,rst=21,busy=26,speed=3,no_flickering=True,inverted=False,dangerous_reaffirm_black=True)
font = MicroFont("victor:R:24.mfnt",cache_chars=True)
@antirez
antirez / resp3.md
Last active June 2, 2020 08:41
RESP3 protocol draft

RESP3 specification

Versions history:

  • 1.0, 2 May 2018, Initial draft to get community feedbacks.

Background

The Redis protocol has served us well in the past years, showing that, if carefully designed, a simple human readable protocol is not the bottleneck in the client server communication, and that the simplicity of the design is a major advantage in creating a healthy client libraries ecosystem.

Yet the Redis experience has shown that after about six years from its introduction (when it replaced the initial Redis protocol), the current RESP protocol could be improved, especially in order to make client implementations simpler and to support new features.

@antirez
antirez / TM.md
Created February 23, 2018 09:04
TripMode TCP out of order bytes reproducing steps

Generate a payload.txt file using this Ruby script:

(1..1000000).each{|i| print "#{i} "}

The file will trivially contain "1 2 3 4 5 ..." so that out of order violations are trivial to spot.

  1. In one terminal open netcat in listen mode: nc -l 6379
  2. In another terminal execute netcat like this: cat payload.txt | nc 127.0.0.1 6379
  3. In the first terminal you should see like 20796 20797 20798 20799 20800... instead of 1 2 3 ...
  4. Does not happen always, so kill the netcat instances, and GOTO 10 to retry until it happens.

Hello!

That's the C function (you can find it inside the Redis source code, inside rax.c).

raxNode *raxRemoveChild(raxNode *parent, raxNode *child) {
    debugnode("raxRemoveChild before", parent);
    /* If parent is a compressed node (having a single child, as for definition
     * of the data structure), the removal of the child consists into turning
     * it into a normal node without children. */
@antirez
antirez / streams_fields.md
Last active February 2, 2018 14:15
Why streams have elements that are actually like hashes?

Why stream items are small hashes instead of single strings like many other Redis types elements is a good question indeed. At the end it's just a design decision, so I don't have the definitive answer. However I can try to explain the design process leading to this design.

What I wanted "Streams" to be, was actually just an Abstract Log. I was not able to call the data structure "log" because it's confusing in many contexts, but that was the idea, and a log better represents what Redis Streams are. Perhaps it's the consumer groups part of the Redis Streams that better characterize the streaming part, but the data structure itself is a log.

Now, what constitutes a log? In the original form, is just lines of text ending with "\n", one after the other, added in an append only fashion. But in general is some data in append only mode.

XADD captures this append only mode of operation. While we have more powerful deletion mechanisms, and will add more, but that is the general idea.

@antirez
antirez / api.md
Last active November 20, 2018 02:52
Consumer groups final API
  • XGROUP CREATE <key> <groupname> <id or $>
  • XGROUP SETID <key> <id or $>
  • XGROUP DELGROUP <key> <groupname>
  • XGROUP DELCONSUMER <key> <consumername>
  • XPENDING <key> [<start> <stop>]
  • XCLAIM <key> <group-name> <consumer-name> <min-idle-time> <ID-1> <ID-2> ...
  • XACK <key> <ID-1> <ID-2> ...
  • XREAD-GROUP (wrapper for XREAD that accepts GROUP and CONSUMER options)
  • XINFO [CONSUMERS|GROUPS|STREAM|...]. STREAM is the default

Meltdown fix impact on Redis performances in virtualized environments

UPDATE: apparently kernel difference may have a serious impact, so I'll redo the test from scratch.

Test performed with AOF enabled, fsync policy 1 second, allowing the rewrites to be triggered.

Command lines used:

#define F float
F f[]={-.1,-.1,1.7,.8,-.11,-6,.4,-.6,.15,-.3,.22,-3.1,-.1,-.06,3.2,.57,-1.1,19,-176,-4.8,-7.2,43,-86,40,48,65,34,803,39,9.65,-17,-33,-33,16,1167,63,-23,41,-22,9,.56,-.08,4.76,-8,-.28,.63,-1182,-7.27,26.6,-1213,198,5.3,-8.25,38.4,.22,-.78,-903,-7.4,-.26,.42,47.5,33.7,36.5,-184,14.5,-90,3};F o[15];F R(F A,F U){o[0]=A;o[1]=U;F a=0;int k=2,m=3,n=0,z=0,l=1;for(int j=0;j<67;j++){if(j==18)n=2,z=3,m=7,l=-17;if(j==60)n=8,l=-59;if((j+l)%m){a+=f[j]*o[n+(j+z)%m];}else{o[k++]=(F)1/(1+exp(-(a+f[j])));a=0;}}return o[14];}main(){for(int i=0;i<2201;i++){putchar(R(i%55,i/55-5)>.5?58:32);if(!(i%55))putchar(10);}}
# Let's add a few entries in our stream "mystream". Every entry in a stream is like
# an hash, composed of fields and values, but every entry can have different fields.
# XADD appends a new entry in a stream, and returns an unique ID which is guaranteed
# be incrementing.
# The first part of the ID is the millisecond unixtime during the
# addition (or the time of the last entry in the stream, if greater than the current time).
# The second part of an entry is a 64 bit counter for entries added in the same ms.
127.0.0.1:6379> XADD mystream name pamela nicknake kill-9
<!DOCTYPE html>
<html>
<head><title>Fizzlefade</title></head>
<body>
<canvas id="framebuffer" width="320" height="200"></canvas>
<script type="text/javascript">
/* Fizzlefade using a Feistel network.