[TOC]
Aggregations are a way to process the results of a search query, group, sort and transform them - and extract analytic insights from them. Much like aggregation queries in other databases and search engines, they can be used to create analytics report, or to perform Faceted Search style queries.
For example, indexing a web-server's logs, we can create report for unique users by hour, country or any other breakdown; or create different reports for errors, warnings, etc.
RediSearch can already index and retrieve large amounts of documents really fast. This is a proposal to allow aggregations on top of that - i.e to enable extracting statitstics and insights from data stored in Redis using RediSearch.
The idea is to perform a search on the RS index, load properties from the fetched documents, and perform calculations based on them - using grouping, sorting, and projection functions. These are composed as a pipeline, and reentrant.
Internally, the aggregation engine uses the same mechanism that loads normal search results (the result processor chain), only using a different set of result processors, which ultimately build result objects from the pipeline. If a normal search processing pipeline looks like filter -> score -> sort -> load documents -> serialize, an aggregation pipeline would look like: filter -> load properties -> group -> reduce -> project -> sort -> serialize.
| #include <redismodule.h> | |
| #include <stdarg.h> | |
| extern RedisModuleCtx *logCtx_g; | |
| // FFS let's have contstants for the logging levels and not hard code them | |
| #define RS_LOG_WARNING "warning" | |
| #define ... | |
| # This script generates mkdocs friendly Markdown documentation from a python package. | |
| # It is based on the the following blog post by Christian Medina | |
| # https://medium.com/python-pandemonium/python-introspection-with-the-inspect-module-2c85d5aa5a48#.twcmlyack | |
| import pydoc | |
| import os, sys | |
| module_header = "# Package {} Documentation\n" | |
| class_header = "## Class {}" | |
| function_header = "### {}" |
A quick guide to write a very very simple "ECHO" style module to redis and load it. It's not really useful of course, but the idea is to illustrate how little boilerplate it takes.
Step 1: open your favorite editor and write/paste the following code in a file called module.c
#include "redismodule.h"
/* ECHO <string> - Echo back a string sent from the client */
int EchoCommand(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
| #ifndef _CRC16_TABLE_H__ | |
| #define _CRC16_TABLE_H__ | |
| /* A table of the shortest possible alphanumeric string that is mapped by redis' crc16 | |
| * to any given redis cluster slot. | |
| * | |
| * The array indexes are slot numbers, so that given a desired slot, this string is guaranteed | |
| * to make redis cluster route a request to the shard holding this slot | |
| */ |