Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save gilangvperdana/1394378d9c340661c46b2f1e65c62980 to your computer and use it in GitHub Desktop.
Save gilangvperdana/1394378d9c340661c46b2f1e65c62980 to your computer and use it in GitHub Desktop.
Step by step to Monitoring OpenStack Instances with Service Discovery Prometheus and Grafana

Monitoring Instances with Prometheus and Grafana

Download Prometheus Server

cd /tmp
wget -c https://github.com/prometheus/prometheus/releases/download/v2.13.1/prometheus-2.13.1.linux-amd64.tar.gz

Extract Prometheus Server

tar zxvf prometheus-2.13.1.linux-amd64.tar.gz

Move to some directory

mv prometheus-2.13.1.linux-amd64 /opt/prometheus-server
cd /opt/prometheus-server

Edit configuration file

  • For example we want to monitor instance on endpoint 167.205.192.12
vim config.yml
# my global config
 
global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).
 
# Alertmanager configuration
alerting:
  alertmanagers:
  - static_configs:
    - targets:
      # - alertmanager:9093
 
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
  # - "first_rules.yml"
  # - "second_rules.yml"
 
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=` to any timeseries scraped from this config.
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090']
  - job_name: node-exporter-instances
    static_configs:
    - targets:
      - 167.205.192.12:9100

Configure Systemd Service for Prometheus Service

vim /etc/systemd/system/prometheus-server.service
[Unit]
Description=Prometheus Server
 
[Service]
User=root
ExecStart=/opt/prometheus-server/prometheus --config.file=/opt/prometheus-server/config.yml --web.external-url=http://ip-server:9090/
 
[Install]
WantedBy=default.target

Running Service

systemctl daemon-reload
systemctl enable prometheus-server.service
systemctl start prometheus-server.service

Verification Service

systemctl status prometheus-server.service
ss -tulpn |grep 9090

Install Grafana

cd /tmp
wget -c https://dl.grafana.com/oss/release/grafana_6.4.3_amd64.deb
dpkg -i grafana_6.4.3_amd64.deb

Running Service

systemctl daemon-reload
systemctl start grafana-server
systemctl enable grafana-server

Verification

systemctl status grafana-server
ss -tulpn | grep 3000

Configura Datasource and Dashboard on Grafana

  • Login to Grafana
  • Add data source
  • Choose Prometheus
Name: Prometheus
Http settings
URL: http://localhost:9090
  • Save and Test
  • Add Dashboard

Create new instance with node exporter

Install node-exporter on ubuntu instances

cd /tmp/
wget -c https://github.com/prometheus/node_exporter/releases/download/v0.18.1/node_exporter-0.18.1.linux-amd64.tar.gz
tar zxvf node_exporter-0.18.1.linux-amd64.tar.gz 
mv node_exporter-0.18.1.linux-amd64 /opt/node_exporter

Create systemd service and running node-exporter

cat << EOF >/etc/systemd/system/node_exporter.service
[Unit]
Description=Node Exporter
 
[Service]
User=root
ExecStart=/opt/node_exporter/node_exporter
 
[Install]
WantedBy=default.target
EOF
 
systemctl daemon-reload
systemctl enable node_exporter.service
systemctl start node_exporter.service
systemctl status node_exporter.service

Best dashboard template

Dashboard Name Grafana ID
Ceph - Cluster 2842
Ceph - Pools 5342
Ceph - OSD (Single) 5336
HAProxy 2428
Servers 11074
Galera/MariaDB 13106
Openstack Dashboard 9701
Docker Monitoring 8321

Best Query for Instance SysLoad Monitoring

avg(node_load5{instance="$node",job="$job"}) /  count(count(node_cpu_seconds_total{instance="$node",job="$job"}) by (cpu)) * 100
100 - (avg by (instance) (rate(node_cpu_seconds_total{job="node",mode="idle"}[1m])) * 100)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment