Label names and values query implementation in Mimir (updated: 2023-07-25)

gistfile1.txt

### Get label names

- Ingester
  - TSDB LabelNames()
    - Head
      - Without matchers:
        - Read from MemPostings -> fast
      - With matchers:
        - PostingsForMatchers() (like any query)
        - LabelNamesFor()
          - Symbolise each series, iterate on the label names and put them into a map (to get the unique set)
    - Block
      - Without matchers:
        - Read from in-memory postings map -> fast
      - With matchers:
        - PostingsForMatchers() (like any query)
        - LabelNamesFor()
          - Symbolise each series, iterate on the label names and put them into a map (to get the unique set)
- Store-gateway
  - Long TTL cache since blocks are immutable
  - Without matchers:
    - Read from in-memory postings map -> fast (assuming the block has already been lazy-loaded)
  - With matchers:
    - Find matching series, symbolise series, iterate on the label names and put them into a map (to get the unique set)

### Label names cardinality

- Ingester (LabelNamesAndValues())
  - Query only the TSDB Head
  - index.LabelNames(matchers...)
  - For each label name:
    - index.LabelValues(labelName, matchers...)
- Store-gateway not queried

### Get series by label matchers

- Ingester
  - MetricsForLabelMatchers()
    - For each matcher:
      - Call q.Select() to find matching series (without fetching chunks)
- Store-gateway
  - Series() with skipChunks=true

### Get label values

- Ingester
  - tsdb.Querier().LabelValues(labelName, matchers...)
  - Head and Block
    - Without matchers:
      - h.head.postings.LabelValues(name)
        - Iterate all label values by label name equal to input (from MemPostings), put them into a map (to get the unique set) -> fast
    - With matchers:
      - PostingsForMatchers(r, matchers...)
      - h.head.postings.LabelValues(name)
      - For each label value returned by LabelValues(name):
        - Call r.Postings(name, value) to get the list of postings for matching label name and value
      - Call index.FindIntersectingPostings() to find the intersection between the result of PostingsForMatchers() and the candidate postings found calling Postings() for each name-value pair
- Store-gateway
  - Long TTL cache since blocks are immutable
  - Without matchers:
    - Find all postings for the label name, querying the index
  - With matchers:
    - Find all postings for the label name, querying the index
    - Either labelValuesFromSeries() or labelValuesFromPostings()
      - labelValuesFromSeries()
        - Run a normal Series() call and then build a map with label values (to get unique set)
      - labelValuesFromPostings()
        - Works similarly to TSDB

### Label values cardinality

- Ingester
  - For each label name in the request
    - Get the label values calling: idxReader.LabelValues(lblName, matchers...)
      - For each label value:
        - Call PostingsForMatchers(), then iterate to count the number of matching series
          - The PostingsForMatchers function can be:
            - in-memory: tsdb.PostingsForMatchers()
            - active: activeseries.NewPostings() wrapping tsdb.PostingsForMatchers()