Skip to content

Instantly share code, notes, and snippets.

@gilangvperdana
Last active July 24, 2022 15:58

Revisions

  1. gilangvperdana revised this gist Jul 24, 2022. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -35,13 +35,13 @@ ingester:
    schema_config:
    configs:
    - from: 2018-04-15
    - from: 2022-07-24
    store: boltdb
    object_store: filesystem
    schema: v11
    index:
    prefix: index_
    period: 168h
    period: 24h
    storage_config:
    boltdb:
  2. gilangvperdana revised this gist May 31, 2022. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -81,6 +81,7 @@ ExecStart=/usr/local/bin/loki -config.file /usr/local/bin/config-loki.yml
    WantedBy=multi-user.target
    ```
    ```
    systemctl enable loki
    sudo service loki start
    sudo service loki status
    ```
    @@ -147,6 +148,7 @@ ExecStart=/usr/local/bin/promtail -config.file /usr/local/bin/config-promtail.ym
    WantedBy=multi-user.target
    ```
    ```
    systemctl enable promtail
    sudo service promtail start
    sudo service promtail status
    ```
  3. gilangvperdana revised this gist May 22, 2022. 1 changed file with 0 additions and 3 deletions.
    3 changes: 0 additions & 3 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -150,9 +150,6 @@ WantedBy=multi-user.target
    sudo service promtail start
    sudo service promtail status
    ```
    ```
    usermod -a -G systemd-journal promtail
    ```

    ## Integrate Grafana with Loki as Data Source
    Create new data source in Grafana with `Loki` and Endpoint `http://127.0.0.1:3100`
  4. gilangvperdana revised this gist May 22, 2022. 1 changed file with 3 additions and 2 deletions.
    5 changes: 3 additions & 2 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -95,8 +95,9 @@ sudo service loki status
    ## Install Promtail Binary and Start a Service
    ```
    cd /usr/local/bin
    sudo curl -fSL -o promtail.gz "https://github.com/grafana/loki/releases/download/v1.6.1/promtail-linux-amd64.gz"
    sudo gunzip promtail.gz
    curl -s https://api.github.com/repos/grafana/loki/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep promtail-linux-amd64.zip | wget -i -
    sudo unzip promtail-linux-amd64.zip
    mv promtail-linux-amd64 promtail
    ```
    ```
    sudo chmod a+x promtail
  5. gilangvperdana revised this gist May 22, 2022. 1 changed file with 4 additions and 3 deletions.
    7 changes: 4 additions & 3 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -7,7 +7,8 @@ curl -s https://api.github.com/repos/grafana/loki/releases/latest | grep browser
    sudo apt install unzip
    sudo unzip loki-linux-amd64.zip
    sudo chmod a+x loki-linux-amd64
    mv loki-linux-amd64 loki
    sudo chmod a+x loki
    ```

    ## Create Loki Config
    @@ -73,7 +74,7 @@ After=network.target
    [Service]
    Type=simple
    User=loki
    User=root
    ExecStart=/usr/local/bin/loki -config.file /usr/local/bin/config-loki.yml
    [Install]
    @@ -138,7 +139,7 @@ After=network.target
    [Service]
    Type=simple
    User=promtail
    User=root
    ExecStart=/usr/local/bin/promtail -config.file /usr/local/bin/config-promtail.yml
    [Install]
  6. gilangvperdana revised this gist May 22, 2022. 1 changed file with 5 additions and 3 deletions.
    8 changes: 5 additions & 3 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -3,9 +3,11 @@
    ## Install Loki Binary and Start as a Service
    ```
    cd /usr/local/bin
    sudo curl -fSL -o loki.gz "https://github.com/grafana/loki/releases/download/v1.6.1/loki-linux-amd64.gz"
    sudo gunzip loki.gz
    sudo chmod a+x loki
    curl -s https://api.github.com/repos/grafana/loki/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep loki-linux-amd64.zip | wget -i -
    sudo apt install unzip
    sudo unzip loki-linux-amd64.zip
    sudo chmod a+x loki-linux-amd64
    ```

    ## Create Loki Config
  7. gilangvperdana revised this gist May 21, 2022. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion README.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Integrate Grafana with Promtail + Loki Provisioning
    # Integrate Grafana with Promtail + Loki

    ## Install Loki Binary and Start as a Service
    ```
  8. gilangvperdana created this gist May 21, 2022.
    182 changes: 182 additions & 0 deletions README.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,182 @@
    # Integrate Grafana with Promtail + Loki Provisioning

    ## Install Loki Binary and Start as a Service
    ```
    cd /usr/local/bin
    sudo curl -fSL -o loki.gz "https://github.com/grafana/loki/releases/download/v1.6.1/loki-linux-amd64.gz"
    sudo gunzip loki.gz
    sudo chmod a+x loki
    ```

    ## Create Loki Config
    ```
    sudo nano config-loki.yml
    ```
    ```
    auth_enabled: false
    server:
    http_listen_port: 3100
    ingester:
    lifecycler:
    address: 127.0.0.1
    ring:
    kvstore:
    store: inmemory
    replication_factor: 1
    final_sleep: 0s
    chunk_idle_period: 5m
    chunk_retain_period: 30s
    max_transfer_retries: 0
    schema_config:
    configs:
    - from: 2018-04-15
    store: boltdb
    object_store: filesystem
    schema: v11
    index:
    prefix: index_
    period: 168h
    storage_config:
    boltdb:
    directory: /tmp/loki/index
    filesystem:
    directory: /tmp/loki/chunks
    limits_config:
    enforce_metric_name: false
    reject_old_samples: true
    reject_old_samples_max_age: 168h
    chunk_store_config:
    max_look_back_period: 0s
    table_manager:
    retention_deletes_enabled: false
    retention_period: 0s
    ```

    ## Configure Loki as a Service
    ```
    sudo nano /etc/systemd/system/loki.service
    ```
    ```
    [Unit]
    Description=Loki service
    After=network.target
    [Service]
    Type=simple
    User=loki
    ExecStart=/usr/local/bin/loki -config.file /usr/local/bin/config-loki.yml
    [Install]
    WantedBy=multi-user.target
    ```
    ```
    sudo service loki start
    sudo service loki status
    ```

    ## Access Loki
    http://[Your-Server-Domain-or-IP]:3100/metrics
    ```
    sudo service loki stop
    sudo service loki status
    ```

    ## Install Promtail Binary and Start a Service
    ```
    cd /usr/local/bin
    sudo curl -fSL -o promtail.gz "https://github.com/grafana/loki/releases/download/v1.6.1/promtail-linux-amd64.gz"
    sudo gunzip promtail.gz
    ```
    ```
    sudo chmod a+x promtail
    ```

    ## Create Promtail Config
    ```
    sudo nano config-promtail.yml
    ```
    ```
    server:
    http_listen_port: 9080
    grpc_listen_port: 0
    positions:
    filename: /tmp/positions.yaml
    clients:
    - url: http://127.0.0.1:3100/loki/api/v1/push
    scrape_configs:
    - job_name: journal
    journal:
    max_age: 12h
    labels:
    job: systemd-journal
    relabel_configs:
    - source_labels: ['__journal__systemd_unit']
    target_label: 'unit'
    ```

    ## Create Promatail Service
    ```
    sudo nano /etc/systemd/system/promtail.service
    ```
    ```
    [Unit]
    Description=Promtail service
    After=network.target
    [Service]
    Type=simple
    User=promtail
    ExecStart=/usr/local/bin/promtail -config.file /usr/local/bin/config-promtail.yml
    [Install]
    WantedBy=multi-user.target
    ```
    ```
    sudo service promtail start
    sudo service promtail status
    ```
    ```
    usermod -a -G systemd-journal promtail
    ```

    ## Integrate Grafana with Loki as Data Source
    Create new data source in Grafana with `Loki` and Endpoint `http://127.0.0.1:3100`

    - In the Log Labels text area, try these LogQL example :
    ```
    # Show All log lines
    {job="systemd-journal"}
    # Show lines containing "info"
    {job="systemd-journal"} |= "info"
    # Show lines containing "error"
    {job="systemd-journal"} |= "error"
    # Show lines containing "info" or "error"
    {job="systemd-journal"} |~ "info|error"
    # Show lines NOT containing "info"
    {job="systemd-journal"} != "info"
    # Show lines including the text "status 403" or "status 503"
    {job="systemd-journal"} |~ "status [45]03"
    ```

    ## Trigger
    To trigger log on Grafana, exec this :
    ```
    echo 'abc123 this is a fake error def678' | systemd-cat
    ```
    Then, monitor on your panel on Grafana.