Skip to content

Instantly share code, notes, and snippets.

View mezzatto's full-sized avatar

Daniel Mezzatto mezzatto

29 followers · 20 following
View GitHub Profile
@mezzatto
mezzatto / dia 1
Last active May 16, 2018 21:42
qconsp2018 - https://qconsp.com/schedule/tabular
performance (netflix) - https://qconsp.com/system/files/keynotes-slides/qcon_sp_keynote_-_martin_spier.pdf
- metricas impressionantes deles (requests, logs, latencia, instancias etc)
- arquitetura (caches nos provedores)
- 3 regioes aws
- ec2 reserved (se nao tá streaming, usa pra recomendacoes, encoding etc)
- build: amis ubuntu
- deploy: 4000 por dia
- tudo stateless
- ferramentas (captura de metricas da redhat, flamegraph, flamescope etc)
@mezzatto
mezzatto / cleanupES.sh
Created August 30, 2016 20:20
ES Cleanup
cd /usr/local/elasticsearch/data/elasticsearch_procura_access/nodes/0/indices
# Remove indices estranhos gerados por requisicoes que nao sao do logstash
# Isso acontece, por exemplo, com sistema de deteccao de vulnerabilidade que a infra roda
echo "Removendo indices estranhos..."
for i in `ls | grep -v ^logstash | grep -v ^ck | grep -v kibana`; do echo -n "Removendo indice" $i ": "; GET -m DELETE http://127.0.0.1:9200/$i; echo ""; done
# Remove indices antigos caso o disco esteja cheio
echo "Removendo indices devido a falta de espaco em disco..."
while [ "`du -sb | cut -f 1`" -gt "50000000000" ]; do idx=`ls -tar | grep -E "(logstash|marvel|ck)" | grep -v kibana | head -1`; echo -n "Removendo indice" $idx ": "; GET -m DELETE http://127.0.0.1:9200/$idx; echo ""; sleep 1; done
@mezzatto
mezzatto / gist:1ce2f5b388a21016b19d4c854538f66c
Created August 24, 2016 14:43
Tracking Update
public class TrackingUpdateRequestProcessor extends UpdateRequestProcessor {
private final SchemaField sf;
public static ConcurrentHashMap<BytesRef, Long> changes = new ConcurrentHashMap<>();
public TrackingUpdateRequestProcessor(SolrQueryRequest req, UpdateRequestProcessor next) {
super(next);
sf = req.getCore().getLatestSchema().getUniqueKeyField();
}
@Override
@mezzatto
mezzatto / Chain
Created August 24, 2016 14:38
Update Chain
<updateRequestProcessorChain name="trackingchain" default="true">
<processor class="solr.LogUpdateProcessorFactory" />
<processor class="solr.DistributedUpdateProcessorFactory" />
<processor class="com.vivareal.solr.plugin.processor.TrackingUpdateRequestProcessorFactory" />
<processor class="solr.RunUpdateProcessorFactory" />
</updateRequestProcessorChain>
@mezzatto
mezzatto / gist:8c18c3ea68bb7e8872f21a5a776636ab
Created August 24, 2016 13:53
CPUs
int nthreads = Math.max(1, Runtime.getRuntime().availableProcessors() / 2);
@mezzatto
mezzatto / readBody.c
Created August 11, 2016 17:02
Read HTTPD request body
#include <strings.h>
#include <sys/types.h>
#include <unistd.h>
#include <zlib.h>
static long int uncompressedBytes(apr_pool_t* p, char** uncompressedData, const void* bytes, size_t length)
{
z_stream strm;
int ret;
unsigned char out[128 * 1024];
@mezzatto
mezzatto / Example output
Created April 8, 2016 20:12
Docket "top"
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O
centauro-prod-1-3-score-8d6e68 0.00% 0 B / 0 B 0.00% 0 B / 0 B 0 B / 0 B
dragon-03232016-street 5.80% 226.9 MB / 3.95 GB 5.74% 0 B / 0 B 50.63 MB / 16.38 kB
dragon-chino-prod-1-3-score-03302016-score-8c292e 8.37% 236.3 MB / 3.95 GB 5.98% 0 B / 0 B 49.5 MB / 12.29 kB
indexer-03232016-street 0.06% 253.7 MB / 3.95 GB 6.42% 574 kB / 152.8 kB 63.71 MB / 12.29 kB
indexer-03302016-score 0.18% 242.6 MB / 3.95 GB 6.14% 137.3 kB / 151.9 kB 63.74 MB / 12.29 kB
locations-monitor-prod 0.00% 70.93 MB / 3.95 GB 1.80% 43
@mezzatto
mezzatto / config_m1
Created September 18, 2014 17:21
bind 172.30.104.126
port 26379
unixsocket "/var/run/redis_sentinel.sock"
daemonize yes
pidfile "/var/run/redis_sentinel.pid"
logfile "/var/log/redis/redis_sentinel.log"
loglevel notice
timeout 240
sentinel monitor Group_4 172.30.104.126 60004 2
@mezzatto
mezzatto / gist:699c38c55354f15cff71
Created September 18, 2014 17:07
[root@redis-lhv-01t ~]# while : ; do ps fawux | grep sentinel | grep -v grep; sleep 1; done
root 56761 8.1 0.0 139812 8656 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.2 0.0 139940 8768 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.2 0.0 140068 8856 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.3 0.0 140068 8956 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.3 0.0 140196 9060 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.3 0.0 140324 9136 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.0 0.0 140452 9224 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.1 0.0 140452 9320 ? Ssl 11:53 0:02 redis-sentinel 172.30.104.126:26379
root 56761 8.0 0.0 140580 9424 ?
@mezzatto
mezzatto / gist:ea6f769c6d5c1eb0a55a
Last active August 29, 2015 14:01
sorted set rate limiter
local ts = tonumber(table.remove(ARGV))
local weight = tonumber(table.remove(ARGV))
local limit = tonumber(ARGV[1])
redis.call('zremrangebyscore', KEYS[1], 0, tonumber(ts) - 60)
local curr = redis.call('zrangebyscore', KEYS[1], tonumber(ts), tonumber(ts))
redis.call('zremrangebyscore', KEYS[1], tonumber(ts), tonumber(ts))
redis.call('zadd', KEYS[1], ts, tonumber(curr[1] or '0') + tonumber(weight))
local all = redis.call('zrange', KEYS[1], 0, -1)