Skip to content

Instantly share code, notes, and snippets.

View filipecosta90's full-sized avatar

Filipe Oliveira (Personal) filipecosta90

View GitHub Profile
@filipecosta90
filipecosta90 / dataloader_snippet.py
Last active March 23, 2019 13:05
redis-porto :: redistimeseries-airquality :: dataloader snippet
# snippet from https://github.com/redis-porto/redistimeseries-airquality/blob/master/dataloader.py
#(...)
#(...)
# redis setup
redis_obj = redis.Redis(host=args.host, port=args.port, password=args.password)
temperature_key = "ts:temperature"
carbon_monoxide_key = "ts:carbon_monoxide"
relative_humidity_key = "ts:relative_humidity"
with open(args.csv, newline="") as csv_file:
@filipecosta90
filipecosta90 / tsrange_snippet.py
Created March 24, 2019 20:27
Snippet with the most relevant part of the complete file, specifically the usage of the Redis command TS.RANGE.
# snippet from https://github.com/redis-porto/redistimeseries-airquality/blob/master/plot.py
#(...)
#(...)
# redis setup
redis_obj = redis.Redis(host=args.host, port=args.port, password=args.password)
temperature_key = "ts:temperature"
temperature_description = "Temperature in °C variation"
temperature_y_label = "°C"
carbon_monoxide_key = "ts:carbon_monoxide"
@filipecosta90
filipecosta90 / list_extend.c
Created March 31, 2019 15:18
list_extend.c -- step 1
/*************************************************************************
*
* Redis 5.X under the hood: 3 - Writing a Redis Module
* ____________________________________________________
*
* [2019] @fcosta_oliveira
*
*/
#include "redismodule.h"
@filipecosta90
filipecosta90 / list_extend.c
Created March 31, 2019 15:50
list_extend.c -- step 2 :: RedisModule_OnLoad
int RedisModule_OnLoad(RedisModuleCtx *ctx, RedisModuleString **argv, int argc) {
if (RedisModule_Init(ctx, "list_extend", 1, REDISMODULE_APIVER_1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
// "write": The command may modify the data set (it may also read from it).
// "deny-oom": The command may use additional memory and should be denied during out of memory conditions.
if (RedisModule_CreateCommand(ctx, "list_extend.filter", ListExtendFilter_RedisCommand, "write deny-oom", 1, 1, 1) == REDISMODULE_ERR)
return REDISMODULE_ERR;
return REDISMODULE_OK;
@filipecosta90
filipecosta90 / list_extend.c
Last active March 31, 2019 16:49
list_extend.c -- step 3 :: ListExtendFilter_RedisCommand
int ListExtendFilter_RedisCommand(RedisModuleCtx *ctx, RedisModuleString **argv,
int argc) {
// LIST_EXTEND.FILTER source_list destination_list low_value high_value
if (argc != 5) {
return RedisModule_WrongArity(ctx);
}
RedisModule_AutoMemory(ctx);
// Open field/value list keys
@filipecosta90
filipecosta90 / imageserver.py
Created April 3, 2019 08:00
RedisGears CDN Sample imageserver.py
#!/usr/bin/env python
"""
Image Server for our CDN RedisGears Example
"""
from wand.image import Image
def convert_to_format( image_binary, fmt ):
converted_bin = None
@filipecosta90
filipecosta90 / script.sh
Created April 3, 2019 08:13
RedisGears CDN Sample - helper script
#!/bin/bash -x
redis-cli flushall
redis-cli module load /Users/filipeoliveira/redis-porto/RedisGears/redisgears.so
redis-cli module load /Users/filipeoliveira/redis-porto/RedisTimeSeries/src/redistimeseries.so
# Register for new cdn events
redis-cli -x rg.pyexecute < imageserver.py
# Load sample images into cdn:* keys
fcosta_oliveira@n1-highcpu-96-redis-server-1:~/STREAM$ ./stream.100M
-------------------------------------------------------------
STREAM version $Revision: 5.10 $
-------------------------------------------------------------
This system uses 8 bytes per array element.
-------------------------------------------------------------
Array size = 100000000 (elements), Offset = 0 (elements)
Memory per array = 762.9 MiB (= 0.7 GiB).
Total memory required = 2288.8 MiB (= 2.2 GiB).
Each kernel will be executed 10 times.
@filipecosta90
filipecosta90 / run-redis-benchmark.sh
Last active April 26, 2019 12:04
script to run redis benchmark with different payload, pipeline, and iothreads, saving the ouput to distinct files
#!/bin/bash
# script to run redis benchmark with different payload, pipeline, and iothreads ( the iothreads are configured on the redis-server VM
# and that argument is only for creating a different file for different redis-server configs
args=("$@")
n=${args[0]}
h=${args[1]}
iothreads=${args[2]}
dir=${args[3]}
@filipecosta90
filipecosta90 / redis_publisher.py
Created November 30, 2019 23:40
Module and script to forward events from wikimedia stream to Redis pub-sub.
#!/usr/bin/env python
"""Module and script to forward events from wikimedia stream to Redis pub-sub."""
import asyncio as aio
import json
import aioredis
from aiohttp_sse_client import client as sse_client
from wikiutils import pubsub_channel, redis_url