Skip to content

Instantly share code, notes, and snippets.

not connected> IDX.CREATE users TYPE HASH SCHEMA tm TIME
OK
127.0.0.1:6379> IDX.INTO users HMSET user1 tm 1476119602
(integer) 1
127.0.0.1:6379> IDX.INTO users HMSET user2 tm 1476019602
(integer) 1
127.0.0.1:6379> IDX.FROM users WHERE "tm < NOW" HGETALL $
1) user2
2) 1) "tm"
@dvirsky
dvirsky / Extensions.md
Last active March 12, 2017 14:34
RediSearch Extensions

Extending RediSearch

RediSearch supports an extension mechanism, much like Redis supports modules. The API is very minimal at the moment, and it does not yet support dynamic loading of extensions in run-time. Instead, extensions must be written in C and compiled into the engine when building it.

There are two kinds of extension APIs at the moment:

  1. Query Expanders, whose role is to expand query tokens (i.e. stemmers).
  2. Scoring Funtions, whose role is to rank search results in query time.

Registering Extensions

@dvirsky
dvirsky / Makefile
Last active June 15, 2017 12:50
Minimal makefile for a redis module
DEBUGFLAGS = -g -ggdb -O2
ifeq ($(DEBUG), 1)
DEBUGFLAGS = -g -ggdb -O0
endif
# find the OS
uname_S := $(shell sh -c 'uname -s 2>/dev/null || echo not')
CFLAGS = -Wall -Wno-unused-function $(DEBUGFLAGS) -fPIC -std=gnu99 -D_GNU_SOURCE
CC:=$(shell sh -c 'type $(CC) >/dev/null 2>/dev/null && echo $(CC) || echo gcc')
#ifndef NS_HEADER_NAME_H_
#define NS_HEADER_NAME_H_
/* Enums are defined like data types */
typedef enum {
/* Options of enums are documented */
NSEnum_Foo,
/* Options of enums are documented */
@dvirsky
dvirsky / tokenize.py
Last active June 28, 2017 08:19
Pre-tokenize source code for searching
import re
import itertools
import sys
def snake_case_split(ident):
"""
Split a snake case identifier into words, returning the original ident and its splits as a list
"""
splits = filter(None, re.split('_', ident))
@dvirsky
dvirsky / redis-bashcomplete
Last active July 20, 2017 00:47
Bash completion for redis server and cli
# bash completion for redis-cli and redis-server
have redis-server &&
_redisserver()
{
local cur prev split=false
COMPREPLY=()
_get_comp_words_by_ref cur prev
/* Helloblock module -- An example of blocking command implementation
* with threads.
*
* -----------------------------------------------------------------------------
*
* Copyright (c) 2016, Salvatore Sanfilippo <antirez at gmail dot com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:

CmdParse - Complex Redis Command Parsing

This is an API to help with parsing complex redis commands - where you need complex and recursive command structues. It can validate and semantically parse commands into a structured AST of sorts.

The main idea is that you can define a Schema for the command, detailing its structure, argument types and so on - and the API can automatically validate and parse incoming redis arguments.

It currently supports:

  • Named and positional arguments.
  • Required and optional arguments.
@dvirsky
dvirsky / RQL.md
Last active December 20, 2017 13:00
Proposed Secondary Index API and Query Language

Proposed Redis Secondary Index Module API

What's This?

Redis is a very sophisticated data structure server, that can be used as a powerful in-memory database. However, if you look at redis as database per-se, it only has primary keys. There is no native way to ask redis for something like "what are the names of users over 18 who have visited my website yesterday?".

If you're familiar with traditional relational databases, this is usually done by creating an index on the relevant columns in your table, allowing to efficiently add complex WHERE clauses to your query. Such an index is called a Secondary Index.

While it is possible (and done by many many people) to implement these indexes on top of redis manually, doing them right and in a performant way is hard.

typedef struct {
RSMultiKey *keys;
} AggregateLoadStep;
typedef struct {
const char *reducer;
CmdArg *args; // TODO: something better here...
const char *alias;
} AggregateGroupReduce;