Dominic Page djptek
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| POST /_bulk | |
| {"create":{"_index":"logs-box_events.events-default"}} | |
| {"agent":{"name":"docker-fleet-agent","id":"352eeec2-76a1-41a2-9ee1-8bae9d9ae980","ephemeral_id":"02cda318-e8e4-4aab-9dbd-eda10d827a5b","type":"filebeat","version":"8.3.0"},"elastic_agent":{"id":"352eeec2-76a1-41a2-9ee1-8bae9d9ae980","version":"8.3.0","snapshot":false},"host":{"hostname":"docker-fleet-agent","os":{"kernel":"5.10.104-linuxkit","codename":"focal","name":"Ubuntu","family":"debian","type":"linux","version":"20.04.4 LTS (Focal Fossa)","platform":"ubuntu"},"containerized":false,"ip":["172.30.0.7"],"name":"docker-fleet-agent","mac":["02:42:ac:1e:00:07"],"architecture":"x86_64"},"data_stream":{"namespace":"default","type":"logs","dataset":"box_events.events"},"message":"{\"source\":null,\"created_by\":{\"type\":\"user\",\"id\":\"2\",\"name\":\"Unknown User\",\"login\":\"\"},\"action_by\":null,\"created_at\":\"2022-10-20T11:38:56-08:00\",\"event_id\":\"97f1b31f-f143-4777-81f8-1b557b39ca33\",\"event_type\":\"SHIELD_ALERT\",\"ip_address\ |
djptek
/ Painless convert_-_to_empty_String.json
Created
July 12, 2022 10:39
Painless Convert "-" to ""
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # test the pipeline using the simulate API | |
| POST _ingest/pipeline/_simulate | |
| { | |
| "docs": [ | |
| { | |
| "_index": "index", | |
| "_source": { | |
| "leave_as_abc": "abc", | |
| "change_to_empty1": "-", | |
| "more_tests": { |
djptek
/ phrase_search_vs_match_only_text.yml
Last active
May 27, 2021 16:56
phrase search vs match_only_text field
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PUT test | |
| { | |
| "mappings": { | |
| "properties": { | |
| "match_only_text": { | |
| "type": "match_only_text" | |
| }, | |
| "text": { | |
| "type": "text" | |
| } |
djptek
/ match_only_field_vs_aggs.json
Created
May 12, 2021 13:16
match_only_field vs aggs on Elasticsearch 8.0 dev build
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # could use kibana but for a quick test simpler to use Elasticsearch standalone with curl | |
| # set the password as a environment variable to make curl easier | |
| export ELASTIC_PWD="<your_elastic_pwd>" | |
| # add mapping | |
| curl -u elastic:${ELASTIC_PWD} -X PUT "localhost:9200/logs?pretty" -H 'Content-Type: application/json' -d' | |
| { | |
| "mappings": { | |
| "properties": { | |
| "@timestamp": { |
djptek
/ nested_nested.json
Last active
April 14, 2021 17:10
nested mapping within nested mapping for Elasticsearch
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PUT nested_nested_test | |
| { | |
| "mappings": { | |
| "properties": { | |
| "1": { | |
| "type": "nested", | |
| "properties": { | |
| "2": { | |
| "type": "nested", | |
| "properties": { |
djptek
/ eng i aggs q
Last active
January 28, 2020 12:57
On which days did we not meet our SLAs (95% of the requests took less than 500ms)?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # On which days did we not meet our SLAs (95% of the requests took less than 500ms)? | |
| GET logs_server1/_search?filter_path=**.key_as_string | |
| { | |
| "size": 0, | |
| "aggs": { | |
| "logs_by_day": { | |
| "date_histogram": { | |
| "field": "@timestamp", | |
| "calendar_interval": "day" |
djptek
/ gist:0b2cfa5a44a08ba2926fecb0a9c6cced
Created
January 17, 2020 15:04
mean of a value per bucket where bucket may have multiple children
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PUT test/_doc/01_01 | |
| { | |
| "parent_id": "01", | |
| "value": 10 | |
| } | |
| PUT test/_doc/01_02 | |
| { | |
| "parent_id": "01", | |
| "value": 10 |
djptek
/ script_query_equal_properties.txt
Last active
December 18, 2019 13:27
script query equal properties
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| PUT test_eq/_doc/test_pass | |
| { | |
| "property1": "foo", | |
| "property2": "foo" | |
| } | |
| PUT test_eq/_doc/test_fail | |
| { | |
| "property1": "foo", | |
| "property2": "bar" |
djptek
/ wait_for_elected_master.sh
Created
September 4, 2019 16:59
Bash snippet to wait for local node to be elected master before proceeding
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| while [ ${NODENAME} != ""`curl -s ${HOSTNAME}:9200/_cat/master\?h=node` ] ; | |
| do echo "waiting for ${NODENAME} to elect itself master" ; | |
| sleep 5 ; | |
| done |
djptek
/ eng_ii_remove_trailing_percent.txt
Last active
April 17, 2020 16:26
recurse over properties map and remove trailing % symbols
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # create a pipline using painless to remove trailing "%" symbols | |
| PUT _ingest/pipeline/remove_percent_symbols | |
| { | |
| "description": "recurse over properties map and remove trailing % symbols", | |
| "processors": [ | |
| { | |
| "script": { | |
| "source": """ | |
| void traverse(Map o) { | |
| for (String childKey : o.keySet()) { |
NewerOlder