Skip to content

Instantly share code, notes, and snippets.

@webmat

webmat/README.md Secret

Last active June 4, 2019 13:40
Show Gist options
  • Save webmat/54a3cfb6a59d1fb512591b110be50491 to your computer and use it in GitHub Desktop.
Save webmat/54a3cfb6a59d1fb512591b110be50491 to your computer and use it in GitHub Desktop.
Download ZIP
How to supercharge the Filebeat Elasticsearch module for support heroes
Raw
README.md

Pipeline for Elasticsearch case logs

This is a procedure you can follow, to automatically ingest Elasticsearch logs from customer cases.

This builds on top of the Filebeat module for Elasticsearch logs. A few adjustments are needed to automatically parse the case ID from the directory name. You can go the extra mile and also add support for Cloud logs, which have an extra header, not supported out of the box by the Filebeat module.

Prior to following this procedure, check out the dashboard screenshots below :-)

Overview

The procedure may seem long, but it's actually not too bad. Here's an overview of what we'll be doing:

  1. Install Filebeat locally
  2. Configure the Filebeat Elasticsearch module
  3. Modify the module's ingest pipeline to tag events with support case IDs and diag bundle names, and to add support for Cloud logs
  4. Import a Kibana dashboard to browse your case logs

Workspace Setup

Create a directory to host your case logs & diagnostics.

In there, you'll create a directory per case, where the directory name starts with the full case number (including 00). For example:

$ ls /Users/mat/elastic/support/
00424241 Acme's thing again
00424242 cluster down

Since we can get more than just ES logs -- and for now this is just a ES log setup -- here's the expectation for where the ES logs will be picked up, under a given case directory:

  • es/*.logs
  • diagnostics*/logs/*.log
  • scrubbed-diagnostics*/logs/*.log

You can adjust these heuristics by modifying the globs in your Filebeat config described below.

Elastic Stack Setup

You'll want to have Elasticsearch and Kibana running. Both can be default configs and local.

Configuring Filebeat

Grab the proper Filebeat package for your OS and install it.

Setup the Elasticsearch module:

./filebeat modules enable elasticsearch
./filebeat setup -e

Edit the module's config modules.d/elasticsearch.yml to set your path globs.

Make sure to adjust the home directory, and the globs to match your directory layout:

server:
  enabled: true
  var.paths:
    - '/Users/mat/work/elastic/support/*/es/*.log'
    - '/Users/mat/work/elastic/support/*/diagnostics-*/logs/*.log'
    - '/Users/mat/work/elastic/support/*/scrubbed-diagnostics-*/logs/*.log'

[Optional] Config to Support Cloud Logs

Step 1 of supporting cloud logs requires adding support for the additional log header. They are prepended by a section such as [es/i-7/es.log] , which is not supported by the Filebeat module for Elasticsearch, but we can fix this.

Edit the module's file module/elasticsearch/server/log.yml and add the optional pattern ({|(\[es/\w-\d+/es\.log\] )? at the beginning of the multiline pattern:

multiline:
    # This original line:
    # pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2})'
    # Should now look like this:
    pattern: '^({|(\[es/\w-\d+/es\.log\] )?\[[0-9]{4}-[0-9]{2}-[0-9]{2})'

Ingest Node Pipeline

Identify the Case ID

The next part is a little hairy, but it's the least invasive way I could find.

We will manually create a small ingest pipeline, then we'll modify the Filebeat pipeline for the Elasticsearch module to call out to our pipeline.

PUT _ingest/pipeline/support_logs
{
  "description" : "Identify Elasticsearch logs per support cases",
  "processors" : [
    {
      "grok" : {
        "field" : "log.file.path",
        "patterns" : [
          "/%{CASE_ID:support.case}"
        ],
        "pattern_definitions" : {
          "CASE_ID" : "00\\d{6}"
        },
        "ignore_failure" : true
      }
    },
    {
      "grok" : {
        "field" : "log.file.path",
        "patterns" : [
          "%{DIAG_LABEL:support.diagnostics}"
        ],
        "pattern_definitions" : {
          "DIAG_LABEL" : """(scrubbed-)?diagnostics-\d{8}-\d{6}"""
        },
        "ignore_failure" : true
      }
    }
  ],
  "on_failure" : [
    {
      "set" : {
        "field" : "ingest.error",
        "value" : "{{ _ingest.on_failure_message }}"
      }
    }
  ]
}

Now for the Hairy Part

Caveat: Any time you upgrade Filebeat and re-run Filebeat setup, you'll need to redo these steps for your new version.

Start preparing the API call to overwrite the ES module's pipeline, don't execute it yet (ajust the version):

PUT _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline

Now grab the current Elasticsearch module's ingest pipeline, to prepare to overwrite it (ajust the version):

GET  _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline
{
  "filebeat-7.0.0-elasticsearch-server-pipeline" : {
    ...
  }
}

Copy out the whole JSON object after the pipeline name "filebeat-7.0.0-elasticsearch-server-pipeline", and paste it in your Kibana console, below the HTTP PUT request we've started preparing. It should look roughly like this:

PUT _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline
{
    "processors" : [
      {
        "rename" : {
          "target_field" : "event.created",
          "field" : "@timestamp"
        }
      },
      
      ...
      
      {
        "remove" : {
          "field" : [
            "first_char"
          ]
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "value" : "{{ _ingest.on_failure_message }}",
          "field" : "error.message"
        }
      }
    ],
    "description" : "Pipeline for parsing elasticsearch server logs"
  }
}

Now add a call out to our custom pipeline, just before the closing square bracket for the processors section:

PUT _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline
{
    "processors" : [
      {
        "rename" : {
          "target_field" : "event.created",
          "field" : "@timestamp"
        }
      },
      
      ...
      
      {
        "remove" : {
          "field" : [
            "first_char"
          ]
        }
      },
      {
        "pipeline" : {
          "name" : "support_logs"
        }
      }
    ],
    "on_failure" : [
      {
        "set" : {
          "value" : "{{ _ingest.on_failure_message }}",
          "field" : "error.message"
        }
      }
    ],
    "description" : "Pipeline for parsing elasticsearch server logs"
  }
}

Execute the API call. Voilà! Our Filebeat Elasticsearch module will now tag Elasticsearch log events with their case ID!

Note: if you point this to a directory containing tons of cases with logs already, they will all be ingested :-) If you don't want this, you can move them somewhere that won't match the globs configured in Filebeat. In my example directory layout, I could simply do this:

mkdir old-cases
mv 00* old-cases/

Finally, we can start Filebeat in tail mode

./filebeat -e

Now create a new case directory, and either extract a diagnostics bundle, or drop Elasticsearch logs under 00000000 my case/es/.

[Optional] Last Step to Support Cloud Logs

You'll have to edit the pipeline "filebeat-7.0.0-elasticsearch-server-pipeline-plaintext" to add support for the additional log header again in the grok parser. Adjus the version ;-)

GET _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline-plaintext

Copy the pipeline body once again, and prepare the API call to overwrite it, adding the CLOUD_SOURCE pattern:

PUT _ingest/pipeline/filebeat-7.0.0-elasticsearch-server-pipeline-plaintext
{
  "description" : "Pipeline for parsing the Elasticsearch server log file in plaintext format.",
  "on_failure" : [
    {
      "set" : {
        "field" : "error.message",
        "value" : "{{ _ingest.on_failure_message }}"
      }
    }
  ],
  "processors" : [
    {
      "grok" : {
        "field" : "message",
        "pattern_definitions" : {
          "GREEDYMULTILINE" : "(.|\n)*",
          "INDEXNAME" : "[a-zA-Z0-9_.-]*",
          "GC_ALL" : """\[gc\]\[%{NUMBER:elasticsearch.server.gc.overhead_seq}\] overhead, spent \[%{NUMBER:elasticsearch.server.gc.collection_duration.time:float}%{DATA:elasticsearch.server.gc.collection_duration.unit}\] collecting in the last \[%{NUMBER:elasticsearch.server.gc.observation_duration.time:float}%{DATA:elasticsearch.server.gc.observation_duration.unit}\]""",
          "GC_YOUNG" : """\[gc\]\[young\]\[%{NUMBER:elasticsearch.server.gc.young.one}\]\[%{NUMBER:elasticsearch.server.gc.young.two}\]%{SPACE}%{GREEDYMULTILINE:message}""",
          "CLOUD_SOURCE" : """es/\w-\d+/es\.log""",
          "LOG_HEADER" : """(\[%{CLOUD_SOURCE:support.cloud_source}\]%{SPACE})?\[%{TIMESTAMP_ISO8601:elasticsearch.server.timestamp}\]\[%{LOGLEVEL:log.level}%{SPACE}?\]\[%{DATA:elasticsearch.component}%{SPACE}\](%{SPACE})?(\[%{DATA:elasticsearch.node.name}\])?(%{SPACE})?"""
        },
        "patterns" : [
          "%{LOG_HEADER}%{GC_ALL}",
          "%{LOG_HEADER}%{GC_YOUNG}",
          """%{LOG_HEADER}%{SPACE}((\[%{INDEXNAME:elasticsearch.index.name}\]|\[%{INDEXNAME:elasticsearch.index.name}\/%{DATA:elasticsearch.index.id}\]))?%{SPACE}%{GREEDYMULTILINE:message}"""
        ]
      }
    }
  ]
}

In the above example for Filebeat 7.0.0, the line

"LOG_HEADER" : """\[%{TIMESTAMP_ISO8601:elasticsearch.server.timestamp}\]\[%{LOGLEVEL:log.level}%{SPACE}?\]\[%{DATA:elasticsearch.component}%{SPACE}\](%{SPACE})?(\[%{DATA:elasticsearch.node.name}\])?(%{SPACE})?"""

Was replaced by 2 lines:

"CLOUD_SOURCE" : """es/\w-\d+/es\.log""",
"LOG_HEADER" : """(\[%{CLOUD_SOURCE:support.cloud_source}\]%{SPACE})?\[%{TIMESTAMP_ISO8601:elasticsearch.server.timestamp}\]\[%{LOGLEVEL:log.level}%{SPACE}?\]\[%{DATA:elasticsearch.component}%{SPACE}\](%{SPACE})?(\[%{DATA:elasticsearch.node.name}\])?(%{SPACE})?"""

You'll notice that the added CLOUD_SOURCE pattern is pretty strict. It reflects what I've seen so far, but it could be made more flexible, if other formats can be seen there.

[Optional] Import Starter Dashboards

At this time, the Elasticsearch module doesn't come with sample dashboards. So I've added support-dashboard.json to this gist, which you can import from Kibana management.

Here's a few pointers to importing them successfully:

  • Prior to importing the dashboards, make sure you've started ingesting logs already, and refresh your filebeat-* index pattern to pick up the custom support.* fields.
  • When importing support-dashboard.json, make sure to re-associate all Kibana Saved Objects with your filebeat-* index pattern.

Dashboard Preview

Customer Elasticsearch Logs - Overview

Overview Dashboard

Customer Elasticsearch Logs - Investigation

Investigation Dashboard

Elasticsearch Events - Saved Search

Saved Search

Raw
support-dashboard.json
[
{
"_id": "8c8c1e10-8223-11e9-a92d-d9f90c518c4f",
"_type": "dashboard",
"_source": {
"title": "Customer Elasticsearch Logs - Investigation [Support]",
"hits": 0,
"description": "",
"panelsJSON": "[{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":33,\"w\":48,\"h\":48,\"i\":\"3\"},\"panelIndex\":\"3\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_0\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":15,\"w\":7,\"h\":12,\"i\":\"4\"},\"panelIndex\":\"4\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_1\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":35,\"y\":15,\"w\":13,\"h\":18,\"i\":\"5\"},\"panelIndex\":\"5\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_2\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":14,\"y\":15,\"w\":21,\"h\":18,\"i\":\"6\"},\"panelIndex\":\"6\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_3\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":0,\"w\":48,\"h\":6,\"i\":\"8\"},\"panelIndex\":\"8\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_4\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":0,\"y\":6,\"w\":48,\"h\":9,\"i\":\"9\"},\"panelIndex\":\"9\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_5\"},{\"embeddableConfig\":{},\"gridData\":{\"x\":7,\"y\":15,\"w\":7,\"h\":12,\"i\":\"10\"},\"panelIndex\":\"10\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_6\"}]",
"optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
"version": 1,
"timeRestore": true,
"timeTo": "now",
"timeFrom": "now-7d",
"refreshInterval": {
"pause": true,
"value": 30000
},
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[{\"$state\":{\"store\":\"appState\"},\"meta\":{\"alias\":\"Errors Only\",\"disabled\":true,\"key\":\"log.level\",\"negate\":false,\"params\":{\"query\":\"ERROR\"},\"type\":\"phrase\",\"value\":\"ERROR\",\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index\"},\"query\":{\"match\":{\"log.level\":{\"query\":\"ERROR\",\"type\":\"phrase\"}}}},{\"$state\":{\"store\":\"appState\"},\"meta\":{\"alias\":null,\"disabled\":true,\"key\":\"support.case\",\"negate\":false,\"params\":{\"query\":\"00424242\"},\"type\":\"phrase\",\"value\":\"00424242\",\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.filter[1].meta.index\"},\"query\":{\"match\":{\"support.case\":{\"query\":\"00424242\",\"type\":\"phrase\"}}}},{\"$state\":{\"store\":\"appState\"},\"meta\":{\"alias\":\"Warnings or Errors\",\"disabled\":true,\"key\":\"log.level\",\"negate\":true,\"params\":[\"ERROR\",\"WARN\"],\"type\":\"phrases\",\"value\":\"ERROR, WARN\",\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.filter[2].meta.index\"},\"query\":{\"bool\":{\"minimum_should_match\":1,\"should\":[{\"match_phrase\":{\"log.level\":\"ERROR\"}},{\"match_phrase\":{\"log.level\":\"WARN\"}}]}}}]}"
}
},
"_migrationVersion": {
"dashboard": "7.0.0"
},
"_references": [
{
"name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index",
"type": "index-pattern",
"id": "filebeat-*"
},
{
"name": "kibanaSavedObjectMeta.searchSourceJSON.filter[1].meta.index",
"type": "index-pattern",
"id": "filebeat-*"
},
{
"name": "kibanaSavedObjectMeta.searchSourceJSON.filter[2].meta.index",
"type": "index-pattern",
"id": "filebeat-*"
},
{
"name": "panel_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_1",
"type": "visualization",
"id": "8e5b26d0-8225-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_2",
"type": "visualization",
"id": "0842c600-8227-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_3",
"type": "visualization",
"id": "719c3ba0-82d5-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_4",
"type": "visualization",
"id": "10d24810-83b0-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_5",
"type": "visualization",
"id": "57264be0-863f-11e9-b1d7-b50bdf2a06e3"
},
{
"name": "panel_6",
"type": "visualization",
"id": "e2a65040-8665-11e9-b1d7-b50bdf2a06e3"
}
]
},
{
"_id": "027b8bd0-8669-11e9-b1d7-b50bdf2a06e3",
"_type": "dashboard",
"_source": {
"title": "Customer Elasticsearch Logs - Overview [Support]",
"hits": 0,
"description": "",
"panelsJSON": "[{\"embeddableConfig\":{},\"gridData\":{\"h\":17,\"i\":\"1\",\"w\":14,\"x\":0,\"y\":6},\"panelIndex\":\"1\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_0\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":6,\"i\":\"2\",\"w\":48,\"x\":0,\"y\":0},\"panelIndex\":\"2\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_1\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":17,\"i\":\"5\",\"w\":17,\"x\":31,\"y\":6},\"panelIndex\":\"5\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_2\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":17,\"i\":\"7\",\"w\":17,\"x\":14,\"y\":6},\"panelIndex\":\"7\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_3\"},{\"embeddableConfig\":{},\"gridData\":{\"h\":15,\"i\":\"8\",\"w\":48,\"x\":0,\"y\":23},\"panelIndex\":\"8\",\"version\":\"7.0.0\",\"panelRefName\":\"panel_4\"}]",
"optionsJSON": "{\"hidePanelTitles\":false,\"useMargins\":true}",
"version": 1,
"timeRestore": true,
"timeTo": "now",
"timeFrom": "now-7d",
"refreshInterval": {
"pause": true,
"value": 30000
},
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"language\":\"kuery\",\"query\":\"\"},\"filter\":[]}"
}
},
"_migrationVersion": {
"dashboard": "7.0.0"
},
"_references": [
{
"name": "panel_0",
"type": "visualization",
"id": "7909c360-8223-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_1",
"type": "visualization",
"id": "db8571a0-8224-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_2",
"type": "visualization",
"id": "0842c600-8227-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_3",
"type": "visualization",
"id": "4f6f5e90-834d-11e9-a92d-d9f90c518c4f"
},
{
"name": "panel_4",
"type": "visualization",
"id": "2fbb6300-8631-11e9-b1d7-b50bdf2a06e3"
}
]
},
{
"_id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f",
"_type": "search",
"_source": {
"title": "Elasticsearch Events [Support]",
"description": "",
"hits": 0,
"columns": [
"support.case",
"elasticsearch.node.name",
"log.level",
"elasticsearch.component",
"message"
],
"sort": [
"@timestamp",
"desc"
],
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"highlightAll\":true,\"version\":true,\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[{\"meta\":{\"alias\":null,\"negate\":false,\"type\":\"phrase\",\"key\":\"event.dataset\",\"value\":\"elasticsearch.server\",\"params\":{\"query\":\"elasticsearch.server\"},\"disabled\":false,\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index\"},\"query\":{\"match\":{\"event.dataset\":{\"query\":\"elasticsearch.server\",\"type\":\"phrase\"}}},\"$state\":{\"store\":\"appState\"}}],\"indexRefName\":\"kibanaSavedObjectMeta.searchSourceJSON.index\"}"
}
},
"_migrationVersion": {
"search": "7.0.0"
},
"_references": [
{
"name": "kibanaSavedObjectMeta.searchSourceJSON.index",
"type": "index-pattern",
"id": "filebeat-*"
},
{
"name": "kibanaSavedObjectMeta.searchSourceJSON.filter[0].meta.index",
"type": "index-pattern",
"id": "filebeat-*"
}
]
},
{
"_id": "719c3ba0-82d5-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "ES Components [Support]",
"visState": "{\"title\":\"ES Components [Support]\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"elasticsearch.component\",\"size\":30,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":true,\"otherBucketLabel\":\"Other\",\"missingBucket\":true,\"missingBucketLabel\":\"Missing\",\"customLabel\":\"ES Component\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "db8571a0-8224-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Events per Case [Support]",
"visState": "{\"title\":\"Events per Case [Support]\",\"type\":\"histogram\",\"params\":{\"type\":\"histogram\",\"grid\":{\"categoryLines\":false},\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"type\":\"category\",\"position\":\"bottom\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\"},\"labels\":{\"show\":true,\"truncate\":100},\"title\":{}}],\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"name\":\"LeftAxis-1\",\"type\":\"value\",\"position\":\"left\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\",\"mode\":\"normal\"},\"labels\":{\"show\":true,\"rotate\":0,\"filter\":false,\"truncate\":100},\"title\":{\"text\":\"Count\"}}],\"seriesParams\":[{\"show\":\"true\",\"type\":\"histogram\",\"mode\":\"stacked\",\"data\":{\"label\":\"Count\",\"id\":\"1\"},\"valueAxis\":\"ValueAxis-1\",\"drawLinesBetweenPoints\":true,\"showCircles\":true}],\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"times\":[],\"addTimeMarker\":false},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"timeRange\":{\"from\":\"now-7d\",\"to\":\"now\"},\"useNormalizedEsInterval\":true,\"interval\":\"auto\",\"time_zone\":\"America/Montreal\",\"drop_partials\":false,\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"support.case\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":true,\"otherBucketLabel\":\"Other\",\"missingBucket\":true,\"missingBucketLabel\":\"Missing\"}}]}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "2fbb6300-8631-11e9-b1d7-b50bdf2a06e3",
"_type": "visualization",
"_source": {
"title": "Log Files [Support]",
"visState": "{\"title\":\"Log Files [Support]\",\"type\":\"table\",\"params\":{\"perPage\":5,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"log.file.path\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":true,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "57264be0-863f-11e9-b1d7-b50bdf2a06e3",
"_type": "visualization",
"_source": {
"title": "ES Components Over Time [Support]",
"visState": "{\"title\":\"ES Components Over Time [Support]\",\"type\":\"histogram\",\"params\":{\"type\":\"histogram\",\"grid\":{\"categoryLines\":false},\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"type\":\"category\",\"position\":\"bottom\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\"},\"labels\":{\"show\":true,\"truncate\":100},\"title\":{}}],\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"name\":\"LeftAxis-1\",\"type\":\"value\",\"position\":\"left\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\",\"mode\":\"normal\"},\"labels\":{\"show\":true,\"rotate\":0,\"filter\":false,\"truncate\":100},\"title\":{\"text\":\"Count\"}}],\"seriesParams\":[{\"show\":\"true\",\"type\":\"histogram\",\"mode\":\"stacked\",\"data\":{\"label\":\"Count\",\"id\":\"1\"},\"valueAxis\":\"ValueAxis-1\",\"drawLinesBetweenPoints\":true,\"showCircles\":true}],\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"times\":[],\"addTimeMarker\":false},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"timeRange\":{\"from\":\"2019-05-31T20:13:08.940Z\",\"to\":\"2019-06-01T20:12:12.363Z\"},\"useNormalizedEsInterval\":true,\"interval\":\"auto\",\"time_zone\":\"America/Montreal\",\"drop_partials\":false,\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"elasticsearch.component\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\"}}]}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "7909c360-8223-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Logs per Support Case",
"visState": "{\"title\":\"Logs per Support Case\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"support.case\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":true,\"otherBucketLabel\":\"Other\",\"missingBucket\":true,\"missingBucketLabel\":\"Missing Case\",\"customLabel\":\"Support Case\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "e2a65040-8665-11e9-b1d7-b50bdf2a06e3",
"_type": "visualization",
"_source": {
"title": "Cloud Sources [Support]",
"visState": "{\"title\":\"Cloud Sources [Support]\",\"type\":\"table\",\"params\":{\"perPage\":5,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"support.cloud_source\",\"size\":50,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\",\"customLabel\":\"Cloud Source\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "8e5b26d0-8225-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Log Level [Support]",
"visState": "{\"title\":\"Log Level [Support]\",\"type\":\"table\",\"params\":{\"perPage\":5,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"log.level\",\"size\":10,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\",\"customLabel\":\"Log level\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "0842c600-8227-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Notes & Nav [Support]",
"visState": "{\"title\":\"Notes & Nav [Support]\",\"type\":\"markdown\",\"params\":{\"fontSize\":12,\"markdown\":\"### Navigation\\n\\n[Overview](#/dashboard/027b8bd0-8669-11e9-b1d7-b50bdf2a06e3)\\n\\n[Investigation](#/dashboard/8c8c1e10-8223-11e9-a92d-d9f90c518c4f)\\n\\n[Saved Search](#/discover/b2dbc750-808e-11e9-a92d-d9f90c518c4f)\\n\\n### Spring Cleaning\\n\\nDelete a case's events\\n```\\nPOST filebeat-*/_delete_by_query\\n{ \\\"query\\\": { \\\"term\\\": {\\n \\\"support.case\\\": {\\n \\\"value\\\": \\\"00424242\\\"\\n}}}}\\n```\",\"openLinksInNewTab\":false},\"aggs\":[]}",
"uiStateJSON": "{}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
}
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": []
},
{
"_id": "4f6f5e90-834d-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Diagnostics Bundles [Support]",
"visState": "{\"title\":\"Diagnostics Bundles [Support]\",\"type\":\"table\",\"params\":{\"perPage\":10,\"showPartialRows\":false,\"showMetricsAtAllLevels\":false,\"sort\":{\"columnIndex\":null,\"direction\":null},\"showTotal\":false,\"totalFunc\":\"sum\"},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"bucket\",\"params\":{\"field\":\"support.diagnostics\",\"size\":20,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":false,\"missingBucketLabel\":\"Missing\"}}]}",
"uiStateJSON": "{\"vis\":{\"params\":{\"sort\":{\"columnIndex\":null,\"direction\":null}}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
},
{
"_id": "10d24810-83b0-11e9-a92d-d9f90c518c4f",
"_type": "visualization",
"_source": {
"title": "Log Levels Over Time [Support]",
"visState": "{\"title\":\"Log Levels Over Time [Support]\",\"type\":\"histogram\",\"params\":{\"type\":\"histogram\",\"grid\":{\"categoryLines\":false},\"categoryAxes\":[{\"id\":\"CategoryAxis-1\",\"type\":\"category\",\"position\":\"bottom\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\"},\"labels\":{\"show\":true,\"truncate\":100},\"title\":{}}],\"valueAxes\":[{\"id\":\"ValueAxis-1\",\"name\":\"LeftAxis-1\",\"type\":\"value\",\"position\":\"left\",\"show\":true,\"style\":{},\"scale\":{\"type\":\"linear\",\"mode\":\"normal\"},\"labels\":{\"show\":true,\"rotate\":0,\"filter\":false,\"truncate\":100},\"title\":{\"text\":\"Count\"}}],\"seriesParams\":[{\"show\":\"true\",\"type\":\"histogram\",\"mode\":\"stacked\",\"data\":{\"label\":\"Count\",\"id\":\"1\"},\"valueAxis\":\"ValueAxis-1\",\"drawLinesBetweenPoints\":true,\"showCircles\":true}],\"addTooltip\":true,\"addLegend\":true,\"legendPosition\":\"right\",\"times\":[],\"addTimeMarker\":false},\"aggs\":[{\"id\":\"1\",\"enabled\":true,\"type\":\"count\",\"schema\":\"metric\",\"params\":{}},{\"id\":\"2\",\"enabled\":true,\"type\":\"date_histogram\",\"schema\":\"segment\",\"params\":{\"field\":\"@timestamp\",\"timeRange\":{\"from\":\"now-7d\",\"to\":\"now\"},\"useNormalizedEsInterval\":true,\"interval\":\"auto\",\"time_zone\":\"America/Montreal\",\"drop_partials\":false,\"customInterval\":\"2h\",\"min_doc_count\":1,\"extended_bounds\":{}}},{\"id\":\"3\",\"enabled\":true,\"type\":\"terms\",\"schema\":\"group\",\"params\":{\"field\":\"log.level\",\"size\":8,\"order\":\"desc\",\"orderBy\":\"1\",\"otherBucket\":false,\"otherBucketLabel\":\"Other\",\"missingBucket\":true,\"missingBucketLabel\":\"Missing\"}}]}",
"uiStateJSON": "{\"vis\":{\"colors\":{\"DEBUG\":\"#DEDAF7\",\"WARN\":\"#F4D598\",\"ERROR\":\"#BF1B00\"}}}",
"description": "",
"version": 1,
"kibanaSavedObjectMeta": {
"searchSourceJSON": "{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"
},
"savedSearchRefName": "search_0"
},
"_migrationVersion": {
"visualization": "7.0.0"
},
"_references": [
{
"name": "search_0",
"type": "search",
"id": "b2dbc750-808e-11e9-a92d-d9f90c518c4f"
}
]
}
]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment