Skip to content

Instantly share code, notes, and snippets.

@gane5h
gane5h / ES-syllabus.md
Last active August 29, 2015 13:59
Polyglot Elasticsearch Workshop Syllabus

Getting started with Elasticsearch

Abstract

This tutorial is an Elasticsearch bootcamp. Elasticsearch is a fully-distributed and scalable search server based on Apache Lucene. Companies like foursquare, soundcloud, github and hundreds more use it to power search and analytics in their applications.

At the end of the day, you’ll:

  1. know the most important concepts and terminology of search engines
  2. have a deep understanding of Elasticsearch
@gane5h
gane5h / gist:ad7453469f6c19276ee5
Created May 23, 2014 08:18
Polyglot Sample Documents
{
"title": "Hackers",
"release_year": 1995,
"genre": ["Action", "Crime", "Drama"],
"actors": ["Johnny Lee Miller", "Angelina Jolie"],
"description": "High-school age computer expert Zero Cool and his hacker friends take on an evil corporation's computer virus with their hacking skills."
}
{
"title": "Johnny Mnemonic",
@gane5h
gane5h / datadog-nginx
Created October 22, 2014 04:06
Nginx log parsing with datadog
"""
Custom parser for nginx log suitable for use by Datadog 'dogstreams'.
To use, add to datadog.conf as follows:
dogstreams: [path to ngnix log (e.g: "/var/log/nginx/access.log"]:[path to this python script (e.g "/usr/share/datadog/agent/dogstream/nginx.py")]:[name of parsing method of this file ("parse")]
so, an example line would be:
dogstreams: /var/log/nginx/access.log:/usr/share/datadog/agent/dogstream/nginx.py:parse
Log of nginx should be defined like that:
log_format time_log '$time_local "$request" S=$status $bytes_sent T=$request_time R=$http_x_forwarded_for';
when starting dd-agent, you can find the collector.log and check if the dogstream initialized successfully
"""
@gane5h
gane5h / story.json
Created December 17, 2015 02:18
Asana Story Example
{
"data": [
{
"created_at": "2015-12-17T02:02:18.858Z",
"created_by": {
"id": 702942504735,
"name": "Ganesh Swami"
},
"id": 74602192326071,
"text": "changed the due date to December 31",
@gane5h
gane5h / mailchimp_export.py
Created June 5, 2017 14:27
Export raw data from Mailchimp lists and campaigns.
'''A python script to export the raw data from mailchimp lists and campaigns.
The output of this script will be a CSV file with the open and click rates for each campaign
and each list member (identified by an email.) You can use this data for further analysis
as seen here:
http://www.silota.com/docs/recipes/sql-email-customer-list-segmentation-lead-scoring.html
Written by Ganesh, 2017.
'''
@gane5h
gane5h / MKR_holders.csv
Created November 30, 2019 23:01
MKR balance storage slot writes
We can't make this file beautiful and searchable because it's too large.
"block_signed_at","address","balance"
"2019-11-30 21:43:30+00","2d1d6d5deae5cc68d893a73781dd573fde302796",1227556400000000000
"2019-11-30 21:43:30+00","2b5634c42055806a59e9107ed44d43c426e58258",239060270600000000000
"2019-11-30 21:33:10+00","39755357759ce0d7f32dc8dc45414cca409ae24e",437140390807426204185
"2019-11-30 21:33:10+00","69076e44a9c70a67d5b79d95795aba299083c275",5878988991879486728884
"2019-11-30 21:33:10+00","e54365c85ca9b7e3ef68c5bed244dc10330bb264",0
"2019-11-30 21:32:32+00","ac8d6652067ca9b33bd15eeeeb01102d2d2cfc64",843100000000000000
"2019-11-30 21:29:27+00","2c4bd064b998838076fa341a83d007fc2fa50957",2964877658410763648961
"2019-11-30 21:29:27+00","9ba4ae48b8ca9e98e511b3371a4d454d1a3e1003",0
"2019-11-30 21:17:29+00","6cc5f688a315f3dc28a7781717a9a798a59fda7b",2633319886385353620901
@gane5h
gane5h / dsr-deposit-volumes.sql
Created January 22, 2020 00:46
Week-on-week DSR deposit volumes
select
date_trunc('week', act_at) as act_at_week,
sum(amount / 1e18) as volume
from (
select
e.block_signed_at as act_at,
'0x' || encode(e.tx_hash, 'hex') as tx_hash,
live.hex_to_int(encode(e.topics[4], 'hex')) as amount,
'DSR_DEPOSIT' as act
from live.block_log_events e