ycombinator

Keybase proof

I hereby claim:

• I am ycombinator on github.
• I am ycombinator (https://keybase.io/ycombinator) on keybase.
• I have a public key whose fingerprint is 88A0 9B75 97D9 3C72 4661 E601 2F4B ECBF E8DD 0F18

To claim this, I am signing this object:

ycombinator

#!/bin/bash

# Find dashboards to fix
to_fix=""
for p in packages/*; do
    pkg=$(echo $p | cut -d"/" -f 2)
    echo "Checking package $pkg..."

    dashboard_filenames=$(find $p/kibana/dashboard -type f 2>/dev/null | cut -d "/" -f 5)
    echo "Dashboards: "

ycombinator

import yaml

def print_yaml(yaml_list):
  print(yaml.safe_dump_all(yaml_list))

def make_yaml_list(tree):
  yaml_list = []
  for key in tree:
    item = {
      "name": key

ycombinator

#!/bin/bash

# Usage:
#   ./find_jenkins_job.sh PR_NUMBER
# Example:
#   ./find_jenkins_job.sh 15790

set -e

NUM_JOBS_TO_SEARCH=100

ycombinator

User perspective

Current state

As of: 7.5.0

• ES:
  • Recommended: User installs and configures MB along side ES. MB monitors ES node and sends monitoring data directly to monitoring cluster, monitoring-* indices.
  • Alternative: User configures ES to send monitoring data to itself (aka production cluster). Production cluster forwards monitoring data to monitoring cluster, .monitoring-* indices.
• Kibana:
• Recommended: User installs and configures MB along side Kibana. MB monitors Kibana instance and sends monitoring data directly to monitoring cluster, monitoring-* indices.

ycombinator

#!/bin/bash

CHILD_PIDFILE=$PWD/child.pid

# Clean up old child, if any
if [ -f $CHILD_PIDFILE ]; then
    OLD_CHILD_PID=$(cat $CHILD_PIDFILE)
    ps $OLD_CHILD_PID >/dev/null
    if [ $? -eq 0 ]; then
        kill -9 $OLD_CHILD_PID

ycombinator

Background

Filebeat consumes Elasticsearch logs via its elasticsearch module. Specifically, for each type of Elasticsearch log (server, gc, deprecation, etc.) there is a corresponding fileset under the Filebeat elasticsearch module. This fileset is responsible for parsing the Elasticsearch log files into structured event that can then be shipped to Elasticsearch or other outputs.

So whenever the structure of Elasticsearch logs changes, the changes must be tested with the Filebeat elasticsearch module to ensure two things:

• that the module can handle the new log structure, and
• that the module can continue to handle the previous log structure (in case a user is running an older version of Elasticsearch against a newer version of Filebeat)

If necessary, the ingest pipeline used by the fileset to do the parsing should be updated.

ycombinator

Request

POST .monitoring-kibana-*/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {

ycombinator

Test if Monitoring Cluster version X is able to monitor Production Cluster version Y, where X > Y.

1. Download and install Elasticsearch version X. This will be the Monitoring Cluster
2. Download and install Elasticearch version Y. This will be the Production Cluster
3. Download and install Kibana version Y. This will be the Production Kibana
4. Start the Monitoring Cluster:

   bin/elasticsearch -E cluster.name=esmon -E node.name=esmon_1 -E http.port=9400
   

5. Start the Production Cluster:

ycombinator