Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
send a dummy alert to prometheus-alertmanager
#!/bin/bash
name=$RANDOM
url='http://localhost:9093/api/v1/alerts'
echo "firing up alert $name"
# change url o
curl -XPOST $url -d "[{
\"status\": \"firing\",
\"labels\": {
\"alertname\": \"$name\",
\"service\": \"my-service\",
\"severity\":\"warning\",
\"instance\": \"$name.example.net\"
},
\"annotations\": {
\"summary\": \"High latency is high!\"
},
\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"
echo ""
echo "press enter to resolve alert"
read
echo "sending resolve"
curl -XPOST $url -d "[{
\"status\": \"resolved\",
\"labels\": {
\"alertname\": \"$name\",
\"service\": \"my-service\",
\"severity\":\"warning\",
\"instance\": \"$name.example.net\"
},
\"annotations\": {
\"summary\": \"High latency is high!\"
},
\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"
echo ""
@jbarr13421

This comment has been minimized.

Copy link

jbarr13421 commented Jun 14, 2018

single quote is your friend..

-d '{
  "alertname": "'$name'"
}'
@Daniel15

This comment has been minimized.

Copy link

Daniel15 commented Jan 25, 2019

This is useful! Thanks for posting it.

@abessifi

This comment has been minimized.

Copy link

abessifi commented Feb 23, 2020

Very useful. Thanks !

@BarakBD-Globality

This comment has been minimized.

Copy link

BarakBD-Globality commented Apr 20, 2020

@msvprogs

This comment has been minimized.

Copy link

msvprogs commented May 26, 2020

Thanks! 👍

@vanugrah

This comment has been minimized.

Copy link

vanugrah commented Jul 23, 2020

To save folks the frustration - this script actually doesn't send the resolve alert according to the v1 api spec. The prometheus contributors have point this out prometheus/alertmanager#1306. The alert only resets after a few minutes because alertmanager expects to get a list of currently firing alerts every 30s-3mins. In order to resolve the alert immediately after the curl you have to include the endsAt timestamp:

url='http://localhost:9093/api/v1/alerts'
echo "Firing up alert" 
curl -XPOST $url -d '[{"status": "firing","labels": {"alertname": "my_cool_alert","service": "curl","severity": "warning","instance": "0"},"annotations": {"summary": "This is a summary","description": "This is a description."},"generatorURL": "http://prometheus.int.example.net/<generating_expression>","startsAt": "2020-07-23T01:05:36+00:00"}]'
echo ""

echo "press enter to resolve alert"
read

echo "sending resolve"
curl -XPOST $url -d '[{"status": "resolved","labels": {"alertname": "my_cool_alert","service": "curl","severity": "warning","instance": "0"},"annotations": {"summary": "This is a summary","description": "This is a description."},"generatorURL": "http://prometheus.int.example.net/<generating_expression>","startsAt": "2020-07-23T01:05:36+00:00","endsAt": "2020-07-23T01:05:38+00:00"}]'
echo ""

Please note you'll have to update the startsAt and endsAt timestamps. I used this site to get RFC3339 compliant timestamps:
https://www.unixtimestamp.com/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.