ycombinator

Install the murmur3 plugin

./bin/elasticsearch-plugin install mapper-murmur3

Add a new field with a multi-field using the murmur3 type

PUT foo/_mapping/bar
{
  "properties": {

ycombinator

diff --git a/gulpfile.js b/gulpfile.js
index e4c4981..5932c25 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -5,7 +5,7 @@ const gulp = require('gulp');
 const g = require('gulp-load-plugins')();
 const path = require('path');
 const del = require('del');
-const isparta = require('isparta');
+// const isparta = require('isparta');

ycombinator

DELETE i
PUT i
{
  "mappings": {
    "t": {
      "properties": {
        "logstash_stats": {
          "type": "object",  
          "properties": {
            "logstash": {

ycombinator

ycombinator

Test if Monitoring Cluster version X is able to monitor Production Cluster version Y, where X > Y.

1. Download and install Elasticsearch version X. This will be the Monitoring Cluster
2. Download and install Elasticearch version Y. This will be the Production Cluster
3. Download and install Kibana version Y. This will be the Production Kibana
4. Start the Monitoring Cluster:

   bin/elasticsearch -E cluster.name=esmon -E node.name=esmon_1 -E http.port=9400
   

5. Start the Production Cluster:

ycombinator

Request

POST .monitoring-kibana-*/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {

ycombinator

Background

Filebeat consumes Elasticsearch logs via its elasticsearch module. Specifically, for each type of Elasticsearch log (server, gc, deprecation, etc.) there is a corresponding fileset under the Filebeat elasticsearch module. This fileset is responsible for parsing the Elasticsearch log files into structured event that can then be shipped to Elasticsearch or other outputs.

So whenever the structure of Elasticsearch logs changes, the changes must be tested with the Filebeat elasticsearch module to ensure two things:

• that the module can handle the new log structure, and
• that the module can continue to handle the previous log structure (in case a user is running an older version of Elasticsearch against a newer version of Filebeat)

If necessary, the ingest pipeline used by the fileset to do the parsing should be updated.

ycombinator

#!/bin/bash

CHILD_PIDFILE=$PWD/child.pid

# Clean up old child, if any
if [ -f $CHILD_PIDFILE ]; then
    OLD_CHILD_PID=$(cat $CHILD_PIDFILE)
    ps $OLD_CHILD_PID >/dev/null
    if [ $? -eq 0 ]; then
        kill -9 $OLD_CHILD_PID

ycombinator

User perspective

Current state

As of: 7.5.0

• ES:
  • Recommended: User installs and configures MB along side ES. MB monitors ES node and sends monitoring data directly to monitoring cluster, monitoring-* indices.
  • Alternative: User configures ES to send monitoring data to itself (aka production cluster). Production cluster forwards monitoring data to monitoring cluster, .monitoring-* indices.
• Kibana:
• Recommended: User installs and configures MB along side Kibana. MB monitors Kibana instance and sends monitoring data directly to monitoring cluster, monitoring-* indices.

ycombinator

#!/bin/bash

# Usage:
#   ./find_jenkins_job.sh PR_NUMBER
# Example:
#   ./find_jenkins_job.sh 15790

set -e

NUM_JOBS_TO_SEARCH=100