Skip to content

Instantly share code, notes, and snippets.

@glortho
Created May 14, 2015 20:29
Show Gist options
  • Save glortho/482a81b860b3c7de7c92 to your computer and use it in GitHub Desktop.
Save glortho/482a81b860b3c7de7c92 to your computer and use it in GitHub Desktop.
Given the keys to the values of the average and standard deviation of a distribution, this transform identifies whether the value of a particular variable is an outlier with respect to that distribution. This enrichment returns a user-specified key with an * anomaly * field that contains _True_ or _False_
#!/bin/sh
"exec" "twxec" "-e" "anomaly_detection" "$0" "$@"
import json
{{ docstring "Given the keys to the values of the average and standard deviation of a distribution, this transform
identifies whether the value of a particular variable is an outlier with respect to that distribution. This enrichment returns
a user-specified key with an * anomaly * field that contains _True_ or _False_." }}
avg_key = {{string Key_name_of_distribution_average_value}}
std_key = {{string Key_name_of_distribution_standard_deviation_value}}
val_key = {{string Key_name_of_variable_value_in_question}}
result_key = {{string Result_key_name}}
def anomaly_detection(msg):
threshold = 2.0 * msg[std_key] + msg[avg_key]
if msg[val_key] > threshold:
msg[result_key] = {"anomaly": True}
else:
msg[result_key] = {"anomaly": False}
return msg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment