Skip to content

Instantly share code, notes, and snippets.

@richcollier
Created July 31, 2020 17:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save richcollier/e03a012b3ebc426e0720f238374ea3ad to your computer and use it in GitHub Desktop.
Save richcollier/e03a012b3ebc426e0720f238374ea3ad to your computer and use it in GitHub Desktop.
#only alert on critical anomalies with a delta of (actual-typical) > X
POST _watcher/watch/_execute
{
"watch": {
"trigger": {
"schedule": {
"interval": "5m"
}
},
"metadata": {
"job_id": "farequote_demo",
"min_record_score": 75,
"min_difference" : 100
},
"input": {
"search": {
"request": {
"indices": [
".ml-anomalies-*"
],
"body": {
"query": {
"bool": {
"filter": [
{
"range": {
"timestamp": {
"gte": "now-5y"
}
}
},
{
"term": {
"result_type": "record"
}
},
{
"term": {
"job_id": "{{ctx.metadata.job_id}}"
}
},
{
"range": {
"record_score": {
"gte": "{{ctx.metadata.min_record_score}}"
}
}
}
]
}
}
}
}
}
},
"condition": {
"script": """
return ctx.payload.hits.hits.stream()
.anyMatch(anomalies -> Math.abs(anomalies._source.actual.0-anomalies._source.typical.0) > ctx.metadata.min_difference);
"""
},
"actions": {
"log": {
"transform": {
"script": """
return ctx.payload.hits.hits.stream()
.filter(anomalies -> Math.abs(anomalies._source.actual.0-anomalies._source.typical.0) > ctx.metadata.min_difference)
.collect(Collectors.toList());"""
},
"logging": {
"text": """
Anomalies:
==========
{{#ctx.payload._value}}
airline={{_source.partition_field_value}} exceeded threshold with actual={{_source.actual.0}} and typical={{_source.typical.0}}
{{/ctx.payload._value}}
"""
}
}
}
}
}
@richcollier
Copy link
Author

richcollier commented Jul 31, 2020

Output

Anomalies:
==========
		airline=AAL exceeded threshold with actual=282.02533259111306 and typical=100.02474685732432
		airline=AAL exceeded threshold with actual=242.3568918440077 and typical=99.80709809346384

@richcollier
Copy link
Author

Change min_difference to 150 and see:

Anomalies:
==========
		airline=AAL exceeded threshold with actual=282.02533259111306 and typical=100.02474685732432

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment