Skip to content

Instantly share code, notes, and snippets.

@radu-gheorghe
radu-gheorghe / logstash.conf
Created March 17, 2016 08:31
logstash grok filter for Elasticsearch logs
filter {
if [type] == "elasticsearch" {
grok {
match => [ "message", "\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{DATA:severity}%{SPACE}\]\[%{DATA:source}%{SPACE}\]%{SPACE}(?<message>(.|\r|\n)*)" ]
overwrite => [ "message" ]
}
if "_grokparsefailure" not in [tags] {
grok { # regular logs
match => [
@radu-gheorghe
radu-gheorghe / securityconfig-secret.yaml
Created September 15, 2023 04:54
Example OpenSearch securityconfig secret for the Kubernetes Operator
apiVersion: v1
kind: Secret
metadata:
name: securityconfig-secret
type: Opaque
stringData:
internal_users.yml: |-
_meta:
type: "internalusers"
config_version: 2
@radu-gheorghe
radu-gheorghe / zookeeper.yaml
Created September 1, 2023 13:24
Zookeeper service for Solr on Kubernetes
apiVersion: v1
kind: Service
metadata:
name: zk-hs
labels:
app: zk
spec:
ports:
- port: 2888
name: server
@radu-gheorghe
radu-gheorghe / pom.xml
Created August 26, 2022 13:45
pom.xml for custom Solr plugin
<!-- NOTE: if you landed here by accident, look for Solr plugin under https://sematext.com/blog and you'll get the context -->
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.example.mysortplugin</groupId>
<artifactId>mysortplugin</artifactId>
@radu-gheorghe
radu-gheorghe / traffic.sh
Created November 16, 2012 14:24
bash script for checking out traffic via /proc/net/dev
#!/bin/bash
#shows traffic on the specified device
function human_readable {
VALUE=$1
BIGGIFIERS=( B K M G )
CURRENT_BIGGIFIER=0
while [ $VALUE -gt 10000 ] ;do
VALUE=$(($VALUE/1000))
@radu-gheorghe
radu-gheorghe / multiple_child_levels.bash
Created April 23, 2013 09:04
multiple parent-child level query with Elasticsearch
curl -XDELETE localhost:9200/test/
curl -XPUT localhost:9200/test/
curl -XPUT localhost:9200/test/son/_mapping -d '{
"son": {
"_parent": {
"type": "mother"
}
}
}'
curl -XPUT localhost:9200/test/daughter/_mapping -d '{
@radu-gheorghe
radu-gheorghe / log_backup.bash
Created July 26, 2012 08:31
Optimize&Backup Elasticsearch index. And restore.
#!/usr/bin/env bash
###############FUNCTIONS############
function prepare {
#optimize the index
echo -n "Optimizing index $INDEX_NAME..."
curl -XPOST "$ADDRESS/$INDEX_NAME/_optimize" 2>/dev/null| grep 'failed":0' >/dev/null
if [ $? -eq 0 ]; then
echo "done"
@radu-gheorghe
radu-gheorghe / count_children.sh
Created November 28, 2012 17:58
Query parents and count children
#cleanup first
curl -XDELETE localhost:9200/test/
echo
#create index
curl -XPUT localhost:9200/test
echo
#create parent type
curl -XPUT localhost:9200/test/test_parent
@radu-gheorghe
radu-gheorghe / sharp_search.sh
Created July 10, 2013 08:43
index and search for a value containing #
curl -XDELETE localhost:9200/test1
curl -XPUT localhost:9200/test1
curl -XPUT localhost:9200/test1/test/_mapping -d '{
"test": {
"properties": {
"foo": {
"type": "string",
"index": "not_analyzed"
}
}
@radu-gheorghe
radu-gheorghe / es_bulk1
Created November 15, 2012 12:11
Elasticsearch bulk insert with index name in URL
$ cat req
{ "index" : { "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "index" : { "_type" : "type1", "_id" : "2" } }
{ "field1" : "value2" }
$ curl -XPOST localhost:9200/testing/_bulk?pretty=true --data-binary @req
{
"took" : 5,
"items" : [ {
"index" : {