Skip to content

Instantly share code, notes, and snippets.

Simple cheatsheet to use elasticsearch

1. Run an elasticsearch single-node instance

docker run -it --rm -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:6.8.3

2. CRUD using cli/curl

@KwangYeol
KwangYeol / websequencediagram.sh
Created June 5, 2017 06:53 — forked from mathbruyen/websequencediagram.sh
Script for fetching sequence diagrams from www.websequencediagrams.com using cURL
#!/bin/sh
# Script for fetching sequence diagrams from www.websequencediagrams.com
# Arguments:
# - 1 (mandatory): text file to be used as input
# - 2 (optional): output file, defaults to input file + format
# Common
format=png
style=napkin
apiVersion=1
@KwangYeol
KwangYeol / partition-in-node.sh
Last active June 8, 2017 07:23
list kafka partitions in node
#!/bin/bash
#
# Usage:
# > bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic my-replicated-topic | ./partition-in-node.sh
#
# Node # 1 2 3 4 5
# Node 01: 12 32 52 72 92
# Node 02: 13 33 53 73 93
# Node 03: 14 34 54 74 94
# Node 04: 15 35 55 75 95
@KwangYeol
KwangYeol / remove_output.py
Last active May 10, 2017 08:37
remove output of the cell from jupyter notebook4
"""
Usage: python remove_output.py notebook.ipynb
Modified from remove_output by Minrk
Modified from remove_output by damianavila
"""
import sys
import io
import os
import nbformat
from urllib.request import urlopen, Request
from urllib.error import HTTPError
from bs4 import BeautifulSoup
import time
import random
import os
import csv
import re
import urllib.parse
@KwangYeol
KwangYeol / scanlog.sh
Created January 20, 2017 02:50
scaffolding for processing daily logs
#!/bin/bash
datediff() {
d1=$(date -d "$1" +%s)
d2=$(date -d "$2" +%s)
if [ ${d1} -gt ${d2} ]; then
echo $(( (d1 - d2) / 86400 ))
else
echo $(( (d2 - d1) / 86400 ))
@KwangYeol
KwangYeol / airflow_daglog.sh
Created January 19, 2017 06:46
grep custom error code from airflow log files.
#!/bin/bash
# custom error message
declare -a errs=("ERR 1" "ERR 2")
# ASSERT: args
if [ $# -ne 3 ]; then
echo "usage: airflow_daglog.sh DAG TASK DS"
echo " ex) airflow_daglog.sh exchage_rate get_exchange_rate 2017-01-1[67]*"
exit $E_BADARGS
#!/bin/bash
# ASSERT: args
if [ $# -ne 1 ]; then
echo "usage: release.sh TITLE"
echo " ex) release.sh working-with-draft"
exit $E_BADARGS
fi
function addExt() {
@KwangYeol
KwangYeol / call_by_name.py
Created January 14, 2017 15:57
Call method using reflection with open/close of transport.
def callit(method_name, *args):
methodToCall = getattr(client, method_name)
client._oprot.trans.open()
result = methodToCall(*args)
client._oprot.trans.close()
return result
@KwangYeol
KwangYeol / hive_method_call.py
Created January 14, 2017 15:55
Hive client methods need opened transport.
client._oprot.trans.open()
dbs = client.get_all_databases()
client._oprot.trans.close()
print(dbs)