Skip to content

Instantly share code, notes, and snippets.

View bvader's full-sized avatar

Stephen Brown bvader

6 followers · 0 following
View GitHub Profile
@bvader
bvader / es-kb-no-security-compose.yml
Created March 3, 2024 19:01
Elasticsearch Plus Kibana 8.x No Security Docker Compose
---
version: '3'
services:
elasticsearch:
container_name: es01
image: docker.elastic.co/elasticsearch/elasticsearch:${TAG}
# 8.x
environment: ['CLI_JAVA_OPTS=-Xms2g -Xmx2g','bootstrap.memory_lock=true','discovery.type=single-node','xpack.security.enabled=false', 'xpack.security.enrollment.enabled=false']
ports:
- 9200:9200
@bvader
bvader / discuss-csv-parsing.txt
Last active January 1, 2024 05:37
For Elastic Discuss Post
https://discuss.elastic.co/t/date-column-has-some-rows-with-null-strict-date-optional-time-causes-exception/350164/6?u=stephenb
PUT _index_template/reams-workorder-logs
{
"index_patterns": [
"reams-workorder-logs-*"
],
"template": {
"settings": {
@bvader
bvader / nginx.yml
Last active December 28, 2023 02:17
# Module: nginx
# Docs: https://www.elastic.co/guide/en/beats/filebeat/main/filebeat-module-nginx.html
- module: nginx
# Access logs
access:
enabled: true
# Set the custom pipeline
input.pipeline: filebeat-8.11.3-nginx-access-pipeline-custom
@bvader
bvader / filebeat-8.11.3-nginx-access-pipeline-custom.json
Created December 28, 2023 02:15
PUT _ingest/pipeline/filebeat-8.11.3-nginx-access-pipeline-custom
{
"description": "Pipeline for parsing Nginx access logs. Requires the geoip and user_agent plugins.",
"processors": [
{
"set": {
"field": "event.ingested",
"value": "{{_ingest.timestamp}}"
}
},
@bvader
bvader / discuss-custom-nginx.log
Last active December 28, 2023 02:13
192.168.0.1 - - [22/Dec/2023:02:54:23 +0000] "MGLNDD_192.168.0.1" 400 166 "-" "-" "-" "test.com" sn="test.com" rt=0.067 ua="-" us="-" ut="-" ul="-" cs=-
192.168.0.2 - - [22/Dec/2023:02:54:36 +0000] "GET /.env HTTP/1.1" 404 197 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36"
192.168.0.3 - - [22/Dec/2023:02:54:37 +0000] "POST / HTTP/1.1" 405 568 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36"
192.168.0.4 - - [22/Dec/2023:14:58:13 +0000] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" "-" "192.168.0.1" sn="test.com" rt=0.000 ua="-" us="-" ut="-" ul="-" cs=-
192.168.0.5 - - [30/Oct/2023:13:17:22 +0000] "POST /test.com/home" 504 578 "https://test.com" "Mozilla/5.0 (X11; CrOS x86_64 14541.0.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36" "-" "test.com" sn="test.com"
@bvader
bvader / es-8.2.3-java-ssl-with-auth.java
Last active June 25, 2022 06:15
Elasticsearch 8.2.3 Java Client with SSL and Authentication
package com.example;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.KeyStore;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
@bvader
bvader / metricbeat-ilm-quickstart.txt
Last active May 26, 2022 15:57
1) Prerequisite : Properly configured Hot / Warm Elasticsearch cluster with correct node attributes.
2) Configure Metricbeat to point directly to Elasticsearch and run setup
NOTE: When metricbeat setup is run, it will overwrite the ILM Policy and recreate the bootstrap index with that policy,
this is often confusing. Running setup also creates all the dashboards, index templates etc. so it is very useful.
As part of the template it will name the ILM policy and rollover_alias in this example metricbeat-7.4.0
./metricbeat setup
@bvader
bvader / elastic-kibana-compose.yml
Created August 15, 2019 17:32
---
version: '2.2'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:${TAG}
environment: ['ES_JAVA_OPTS=-Xms2g -Xmx2g','bootstrap.memory_lock=true','discovery.type=single-node', 'http.host=0.0.0.0', 'transport.host=127.0.0.1']
ports: ['127.0.0.1:9200:9200']
networks: ['stack']
ulimits:
memlock:
@bvader
bvader / PCF-Space-Drain-ELK-Stack-Quick-Start.txt
Last active September 11, 2019 15:56
Quick Start for PCF, Space Drain and ELK Stack
##
# Assumes Basic Understanding of PCF, Elasticsearch, Kibana and logstash
# Should support Elasticstack 7.X
##
##
# Step 1: Setup index template
# Save this file https://gist.github.com/bvader/addf80083b170e0cfcd78f946a78d50e
# to pcf_space_drain_log_template.json
# Then run the following command replacing the username, password and elasticearchhost
@bvader
bvader / pcf-rtr-logs-saved-objects-7x.json
Last active September 7, 2019 01:39
PCF RTR Logs Kibana Saved Objects
[
{
"_id": "af6e3990-870a-11e9-808a-77559368a0be",
"_type": "dashboard",
"_source": {
"title": "pcf-rtr-dashboard",
"hits": 0,
"description": "",
"panelsJSON": "[{\"gridData\":{\"x\":0,\"y\":0,\"w\":24,\"h\":15,\"i\":\"1\"},\"version\":\"7.0.0\",\"panelIndex\":\"1\",\"embeddableConfig\":{},\"panelRefName\":\"panel_0\"},{\"gridData\":{\"x\":24,\"y\":0,\"w\":24,\"h\":15,\"i\":\"2\"},\"version\":\"7.0.0\",\"panelIndex\":\"2\",\"embeddableConfig\":{},\"panelRefName\":\"panel_1\"},{\"gridData\":{\"x\":0,\"y\":46,\"w\":48,\"h\":15,\"i\":\"3\"},\"version\":\"7.0.0\",\"panelIndex\":\"3\",\"embeddableConfig\":{},\"panelRefName\":\"panel_2\"},{\"gridData\":{\"x\":0,\"y\":15,\"w\":48,\"h\":15,\"i\":\"4\"},\"version\":\"7.0.0\",\"panelIndex\":\"4\",\"embeddableConfig\":{},\"panelRefName\":\"panel_3\"},{\"gridData\":{\"x\":0,\"y\":30,\"w\":48,\"h\":16,\"i\":\"5\"},\"version\":\"7.0.0\",\"panelIndex\":\"5\",\"embeddableConfig\":{},\"panelRefName\":\"panel_4\"}]",
"optionsJSON": "