Systemd Alert to Telegram
The purposes is to auto alerting while systemd goes to failed/activate/stopped state to telegram with efficient scripting & services.
Warn
your_service
is your systemd service who you want to monitor.
Configure
- Set Telegram Key
mkdir /etc/telegram
nano /etc/telegram/key.sh
- TELEGRAM_BOT_KEY you can get from BOT_FATHER on telegram
- CHAT_ID you can get from
https://api.telegram.org/botYOUR_TELEGRAM_BOT_KEY/getUpdates
export TELEGRAM_KEY="TELEGRAM_BOT_KEY"
export CHAT_ID="-1001795547276"
chmod +x /etc/telegram/key.sh
- Create Telegram Service
nano /usr/bin/telegram
#!/bin/bash
source /etc/telegram/key.sh
URL="https://api.telegram.org/bot5136513143:AAGscRDJ_t5W3Al54vlhbLxnXBl52WD85lQ/sendMessage"
curl -s -d "chat_id=$CHAT_ID&disable_web_page_preview=1&text=$1" $URL > /dev/null
chmod +x /usr/bin/telegram
- Create Script for All State of Services, we can make Activating|Failed|Stopped State in one Script
nano /usr/bin/your_service-status-allinone-telegram
#!/bin/bash
shopt -s nocasematch
UNIT=your_service_you_want_to_monitor
status=$(systemctl is-active $UNIT)
HOSTNAME=$(hostname)
IPPRIV=$(ifconfig ens3 | grep "inet " | awk '{print $2}')
UNITSTATUS=$(systemctl status $UNIT | grep 'Active')
ALERT=$(echo -e "\u26A0")
DATE=$(date)
case $status in
active)
telegram "$ALERT Alert Report $UNIT Active $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
$UNIT currently in active state"
;;
activating)
telegram "$ALERT Alert Report Elasticsearch Activating $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
$UNIT currently in activating state"
;;
loaded)
telegram "$ALERT Alert Report Elasticsearch Loaded $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
$UNIT currently in loaded state"
;;
failed)
telegram "$ALERT Alert Report Elasticsearch Failed $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
Ops, $UNIT is on failed state. Please check cluster now!"
;;
deactivating)
telegram "$ALERT Alert Report Elasticsearch Deactivating $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
Ops, $UNIT is on deactivating state. Please check cluster now!"
;;
inactive)
telegram "$ALERT Alert Report Elasticsearch Inactive $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Message:
$UNITSTATUS
Ops, $UNIT is on Inactive state. Please check cluster now!"
;;
esac
chmod +x /usr/bin/your_service-status-allinone-telegram
- Make Script can Execute from Systemd-Journald
nano /etc/systemd/system/your_service-status-failed-telegram@.service
[Unit]
Description=Unit Status Telegram Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/your_service-status-allinone-telegram %I
- Declare to Systemd Service, you can check the systemd directory from
systemctl status your_service
[Unit]
OnFailure=your_service-status-allinone-telegram@.service
[Service]
ExecStopPost=+/usr/bin/your_service-status-allinone-telegram
ExecStartPost=+/usr/bin/your_service-status-allinone-telegram
chmod +x /etc/systemd/system/your_service-status-failed-telegram@.service
- Daemon Reload
systemctl daemon-reload
- Test
systemctl restart your_service
systemctl stop your_service
systemctl start your_service
[UPDATE] Efficiency Systemd Alert to Telegram
This version just use one systemd service and script for all service who want to monitor. The difference is only in the addition of parameters after command execution, eg ./systemd-alert your_service
Warn
your_service
is your systemd service who you want to monitor.
Configure
- Set Telegram Key
mkdir /etc/telegram
nano /etc/telegram/key.sh
- TELEGRAM_BOT_KEY you can get from BOT_FATHER on telegram
- CHAT_ID you can get from
https://api.telegram.org/botYOUR_TELEGRAM_BOT_KEY/getUpdates
export TELEGRAM_KEY="TELEGRAM_BOT_KEY"
export CHAT_ID="-1001795547276"
chmod +x /etc/telegram/key.sh
- Create Telegram Service
nano /usr/bin/telegram
#!/bin/bash
source /etc/telegram/key.sh
URL="https://api.telegram.org/bot5136513143:AAGscRDJ_t5W3Al54vlhbLxnXBl52WD85lQ/sendMessage"
curl -s -d "chat_id=$CHAT_ID&disable_web_page_preview=1&text=$1" $URL > /dev/null
chmod +x /usr/bin/telegram
- Create Script for All State of Services, we can make Activating|Failed|Stopped State in one Script
nano /usr/bin/alerting-systemd
#!/bin/bash
shopt -s nocasematch
UNIT=$1
status=$(systemctl is-active $UNIT)
HOSTNAME=$(hostname)
IPPRIV=$(ifconfig ens3 | grep "inet " | awk '{print $2}')
IPPUB=$(curl ifconfig.me)
UNITSTATUS=$(systemctl status $UNIT | grep 'Active')
ALERT=$(echo -e "\u26A0")
DATE=$(date)
case $status in
active)
telegram "[Active] $UNIT on Active State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
$UNIT currently in active state"
;;
activating)
telegram "[ACTIVATING] $UNIT on Activating State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
$UNIT currently in activating state"
;;
loaded)
telegram "[LOADED] $UNIT on Loaded State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
$UNIT currently in loaded state"
;;
failed)
telegram "[FAILED] $UNIT on Failed State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
Ops, $UNIT is on failed state. Please check cluster now!"
;;
deactivating)
telegram "[DEACTIVATING] $UNIT on Deactivatiing State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
Ops, $UNIT is on deactivating state. Please check cluster now!"
;;
inactive)
telegram "[INACTIVE] $UNIT on Inactive State $ALERT
Date: $DATE
Host: $HOSTNAME
IP: $IPPRIV
Public IP: $IPPUB
Message:
$UNITSTATUS
Ops, $UNIT is on Inactive state. Please check cluster now!"
;;
esac
chmod +x /usr/bin/alerting-systemd
- Make Script can Execute from Systemd-Journald
nano /etc/systemd/system/alerting-systemd@.service
[Unit]
Description=Unit Status Telegram Service
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/alerting-systemd $1 %I
chmod +x /etc/systemd/system/alerting-systemd@.service
- Declare to Systemd Service, you can check the systemd directory from
systemctl status your_service
[Unit]
OnFailure=alerting-systemd@.service `your_service`
[Service]
ExecStopPost=+/usr/bin/alerting-systemd `your_service`
ExecStartPost=+/usr/bin/alerting-systemd `your_service`
- Daemon Reload
systemctl daemon-reload
- Test
systemctl restart your_service
systemctl stop your_service
systemctl start your_service