andrewkroh

processors:
  - script:
      # This uses a Beat script processor to include only ipv4 addresses
      # in the host.ip field. This would need to placed after the add_host_metadata
      # processor.
      #
      # It would be a lot more efficient to have add_host_metadata allow controlling
      # what addresses were included because this has to execute for every event.
      #
      # References:

andrewkroh

Adding event.ingested and lag calculations to Winlogbeat events

Create an Ingest Pipeline that will add four fields:

• event.ingested - Time when the event was processed by Elasticsearch.
• event.lag.read - Time difference in milliseconds between @timestamp and event.created. This measures how long it took for Winlogbeat read the event from the event log (for WEC this includes the delivery time from forwarder to collector).
• event.lag.ingest - Time difference in milliseconds between event.created and event.ingested. This measures the time between Winlogbeat reading the event (time when it "created" the document) to when it was written to Elasticsearch.

andrewkroh

DRAFT: Routing Filebeat data to a Fleet integration data stream

This is an unofficial tutorial that may be useful to users that are in the process of migrating to to Elastic Agent and Fleet. It explains the steps to route some Filebeat data into a data stream managed by a Fleet integration package.

Install the Fleet integration

Installing a Fleet integration sets up all of its data streams and dashboards. There are two methods to install. In these examples we install the Hashicorp Vault 1.3.1 integration.

Use Kibana (easiest)

andrewkroh

#!/usr/bin/env bash
# Licensed to Elasticsearch B.V. under one or more contributor
# license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright
# ownership. Elasticsearch B.V. licenses this file to you under
# the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http:#www.apache.org/licenses/LICENSE-2.0

andrewkroh

{
  "@timestamp": "2019-01-29T19:10:47.538Z",
  "beat": {
    "hostname": "DESKTOP",
    "name": "DESKTOP",
    "version": "6.3.2"
  },
  "event": {
    "kind": "event"
  },

andrewkroh

package main

import (
	"flag"
	"log"
	"os/user"
	"syscall"
	"unsafe"

	"golang.org/x/sys/windows"

andrewkroh

winlogbeat.event_logs:
  - name: Security
    ignore_older: 1h
    processors:
      - script:
          lang: javascript
          source: |
            var console = require("console");

            var ids = {

andrewkroh

Elastic Beats Development 101

This is a short guide to get up and building Elastic Beats on a new Linux host.

Start a VM

This uses Google Compute Engine (GCE) to start an Ubuntu 20.04 virtual machine. You can use other versions of Linux or different virtualization platforms (or no virtualization), but those are not guaranteed to work with the commands here.

 gcloud auth login

andrewkroh

Id          : 1
Version     : 0
LogLink     : System.Diagnostics.Eventing.Reader.EventLogLink
Level       : System.Diagnostics.Eventing.Reader.EventLevel
Opcode      : System.Diagnostics.Eventing.Reader.EventOpcode
Task        : System.Diagnostics.Eventing.Reader.EventTask
Keywords    : {, fi:FileNameCreate}
Template    : <template xmlns="http://schemas.microsoft.com/win/2004/08/events">

andrewkroh

var processor = require("processor");

var filebeatCisco = (function() {
    var parseCiscoHeader = new processor.Dissect({
        "tokenizer": "%{}%%{cisco.log.facility}-%{cisco.log.severity}-%{event.code}: %{message}",
        "field": "log.original",
        "target_prefix": "",
    }).Run;

    var coerceDataTypes = new processor.Transform([