prometheus - scrape multiple containers in a pod

gistfile1.txt

    # Example scrape config for pods
    #
    # The relabeling allows the actual pod scrape endpoint to be configured via the
    # following annotations:
    #
    # * `prometheus.io/scrape`: Only scrape pods that have a value of `true`
    # * `prometheus.io/path`: If the metrics path is not `/metrics` override this. This
    #    will be the same for every container in the pod that is scraped.
    # * this will scrape every container in a pod with `prometheus.io/scrape` set to true and the
        port is name `metrics` in the container
    # * note `prometheus.io/port` is no longer honored. You must name the port(s) to scrape `metrics`
    #   Also, in some of the issues I read, there was mention of a container role, but I couldn't get 
    #   that to work - or find any more info on it.
    - job_name: 'kubernetes-pods'

      kubernetes_sd_configs:
      - role: pod

      relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_container_port_name]
        action: keep
        regex: metrics
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [ __address__, __meta_kubernetes_pod_container_port_number]
        action: replace
        regex: (.+):(?:\d+);(\d+)
        replacement: ${1}:${2}
        target_label: __address__
      - action: labelmap
        regex: __meta_kubernetes_pod_label_(.+)
      - source_labels: [__meta_kubernetes_namespace]
        action: replace
        target_label: kubernetes_namespace
      - source_labels: [__meta_kubernetes_pod_name]
        action: replace
        target_label: kubernetes_pod_name

I now use regex: metrics(-.*)? to match all ports that begin with metrics- while still matching just "metrics"

Hi,
Which file should this : relabel_configs be put ?? Thanks in advance

@bakins how do you deal with two different ports in two different paths?

Different schemes could be dealt with by naming the ports http-metrics / https-metrics and adjusting the relabel configs accordingly, but different paths are a problem.

Also what happens if you have identical metric names from two different ports within the same pod?