[Influxdb Cardinality Debug] #influxdb

influxdb.md

• SHOW TAG VALUES EXACT CARDINALITY WITH KEY = "index";
• SHOW SERIES CARDINALITY group by "measurement"; → Important useful one.
• influxdata/influxdb#8636 → Good examples are here
• influx_inspect report -detailed /var/lib/influxdb/data/envoy/autogen/903 The best way to ficure out high cardinality series.
• figure out series in each database use _internal ; select * from "database" where time > now() - 900s limit 10; Faced this error ERR: cannot delete data. DB contains shards using both inmem and tsi1 indexes. Please convert all shards to use the same index type to delete data.

find /var/lib/influxdb -type d -name index | while read index; do rm -Rf "$index" ; done
influx_inspect buildtsi -database envoy -datadir /var/lib/influxdb/data/ -waldir /var/lib/influxdb/wal
influx_inspect buildtsi -database telegraf -datadir /var/lib/influxdb/data/ -waldir /var/lib/influxdb/wal

chown -R influxdb:influxdb /var/lib/influxdb/

• influx_inspect verify -dir /var/lib/influxdb
• Use backup and restore of influxdb to migrate to new machines. it support time range as well. Influxdb use

max-series-per-database = 0
max-values-per-tag = 0 
index-version = "tsi1"

tune the above values.

Fo io tuning

# The amount of time that a write will wait before fsyncing.  A duration
  # greater than 0 can be used to batch up multiple fsync calls.  This is useful for slower
  # disks or when WAL write contention is seen.  A value of 0s fsyncs every write to the WAL.
  # Values in the range of 0-100ms are recommended for non-SSD disks.
  # wal-fsync-delay = "0s"
  
  
  max-select-point = 1000
  log-queries-after = "45s
  query-timeout = "120s"
  max-concurrent-queries = 25

Putting a limit at micro service level using one db per nomad-task-name

• https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb
• database_tag in telegraf output.
• content_encoding this to gzip

Towards multi influxdb database instances.

• influxdata/telegraf#1778 (comment)
• Read this https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md#output-configuration-examples
• database_tag in influxdb to route metrics multiple databases in the same instance.