Skip to content

Instantly share code, notes, and snippets.

@l13t
Created January 16, 2019 13:21
Show Gist options
  • Save l13t/d432b63641b6972b1f58d7c037eec88f to your computer and use it in GitHub Desktop.
Save l13t/d432b63641b6972b1f58d7c037eec88f to your computer and use it in GitHub Desktop.
alertmanager alerts to slack for prometheus-operator
.......some config.....
alertmanager:
## Deploy alertmanager
##
enabled: true
## Service account for Alertmanager to use.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
##
serviceAccount:
create: true
name: ""
## Configure pod disruption budgets for Alertmanager
## ref: https://kubernetes.io/docs/tasks/run-application/configure-pdb/#specifying-a-poddisruptionbudget
## This configuration is immutable once created and will require the PDB to be deleted to be changed
## https://github.com/kubernetes/kubernetes/issues/45398
##
podDisruptionBudget:
enabled: false
minAvailable: 1
maxUnavailable: ""
## Alertmanager configuration directives
## ref: https://prometheus.io/docs/alerting/configuration/#configuration-file
## https://prometheus.io/webtools/alerting/routing-tree-editor/
##
config:
global:
resolve_timeout: 5m
slack_api_url: "<slack_url>"
route:
group_by: ['job']
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
#receiver: 'slack'
routes:
- match:
alertname: DeadMansSwitch
receiver: 'null'
- match:
receiver: 'slack'
continue: true
receivers:
- name: 'null'
- name: 'slack'
slack_configs:
- channel: '#alertmanager-live'
send_resolved: false
title: '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] Monitoring Event Notification'
text: >-
{{ range .Alerts }}
*Alert:* {{ .Annotations.summary }} - `{{ .Labels.severity }}`
*Description:* {{ .Annotations.description }}
*Graph:* <{{ .GeneratorURL }}|:chart_with_upwards_trend:> *Runbook:* <{{ .Annotations.runbook }}|:spiral_note_pad:>
*Details:*
{{ range .Labels.SortedPairs }} • *{{ .Name }}:* `{{ .Value }}`
{{ end }}
{{ end }}
.......some config.....
@bayucandra
Copy link

Thanks for sharing, just copy paste and adjust slack_url, it really works. Nice work.

@pushp1997
Copy link

pushp1997 commented Nov 5, 2020

Hey @bayucandra can you share the entire yaml with the ".......some config....."? How do I use this? is it a deployment or stateful or service yaml?

I have deployed Prometheus Operator in my cluster

@dharmendrakariya
Copy link

Hi @pushp1997 its a portion of values.yaml file which you will find under your helm chart repo ( I hope you have used helm to install Prometheus Operator, in which case only you need this values.yaml file).

@anoopl
Copy link

anoopl commented Apr 28, 2021

Does this work with Kube-prometheus https://github.com/prometheus-operator/kube-prometheus

@anoopl
Copy link

anoopl commented Apr 29, 2021

I got the same working on Kube-prometheus

@dnaranjor
Copy link

Hey @anoopl where in kube-prometheus manifests is this change applied to make it work? I have the same situation...

@anoopl
Copy link

anoopl commented Jun 28, 2021

@dnaranjor on values.yaml you can see alertmanger: under that you can add something like below
config:
global:
resolve_timeout: 5m

@dnaranjor
Copy link

@anoopl, thing is, I'm not using the Helm chart thingy, I applied manually the YAMLs under /manifest folder to my cluster, and I can't see that vaules.yaml file aywhere... :-(

@anoopl
Copy link

anoopl commented Jun 29, 2021

@dnaranjor You will have values.yaml only when you install it with Helm.
This configs should be some where else in manifest, I am not sure, have not tried the manual installation

@somaninga-pujari
Copy link

great! was so helpful :-)

@jaikrishnaa
Copy link

how to test it

I got the same working on Kube-prometheus

doesnt work for me @anoopl

@sadath-12
Copy link

Thanks for sharing, just copy paste and adjust slack_url, it really works. Nice work.

u create an app and recieve for ex @app right ? @bayucandra

@samnzay
Copy link

samnzay commented May 24, 2023

It works well. thanks, man

@AnaisUrlichs
Copy link

Thank you thank you thank you

@HiraoManato
Copy link

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment