Skip to content

Instantly share code, notes, and snippets.

@Pandry
Last active December 4, 2019 15:41
Show Gist options
  • Save Pandry/5aafc90889608344b58c1e4933cb1a11 to your computer and use it in GitHub Desktop.
Save Pandry/5aafc90889608344b58c1e4933cb1a11 to your computer and use it in GitHub Desktop.
A script to count all the point in a InfluxDB database
#!/bin/bash
##
# Author: github.com/Pandry
# Version: v1.0
##
INFLUXURL="http://127.0.0.1:8086"
#name of the database
DBNAME="telegraf-test"
#host is the tag present on every telegraf measurement
# If TAGS is unset, it will not group by tags the query (also the extraction will be faster)
TAGS=("host")
#Name of the reporting document
DOC_NAME="report.csv"
#The base CURL command
CURL_BASE_CMD="curl -s \"$INFLUXURL/query?pretty=true\" --data-urlencode \"db=$DBNAME\""
MEASUREMENTS_REGEX="/EngineData.*/"
#Elaborate eventual tags
QUERY_END="GROUP BY "
CSV_GROUPS=""
COMMAS=""
if [[ "${#TAGS[*]}" -eq "0" ]]; then
unset QUERY_END
else
for e in ${TAGS[*]}; do
QUERY_END+="\"$e\","
CSV_GROUPS+="$e;"
COMMAS+=";"
done
QUERY_END=`echo $QUERY_END | rev | cut -c 2- | rev`
fi
echo "measurement;datapoints;${CSV_GROUPS}time;comment" > $DOC_NAME
if [[ $MEASUREMENTS_REGEX != "" ]]; then
M_QUERY_END="WITH MEASUREMENT =~ $MEASUREMENTS_REGEX"
fi
#Get the header from the DB
#Query for the measurements names
USEDTIME=$(eval $(echo "$CURL_BASE_CMD --data-urlencode 'q=SHOW MEASUREMENTS $M_QUERY_END' -w %{time_total} -o /tmp/measurements.json"))
#Get the number of measurements
MEASUREMENTS_LEN=`cat /tmp/measurements.json | jq ".results[0].series[0].values | length " `
#Adding request to stats
echo "\$measurements;$MEASUREMENTS_LEN;${COMMAS}$USEDTIME;Query used to request the measurements" >> $DOC_NAME
#In case a limit is needed
#MEASUREMENTS_LEN=10
#Looping for each measurement
for (( i=0; i<=$(($MEASUREMENTS_LEN - 1 )); i++ )); do
CURRENT_MEASUREMENT_NAME=`cat /tmp/measurements.json | jq -r ".results[0].series[0].values[$i][0]" `
echo "Doing measurement [$(($i+1))/$MEASUREMENTS_LEN] $CURRENT_MEASUREMENT_NAME "
USEDTIME=$(eval $(echo "$CURL_BASE_CMD --data-urlencode 'q=SELECT COUNT(*) FROM \"$CURRENT_MEASUREMENT_NAME\" $QUERY_END' -w %{time_total} -o /tmp/tmpdata.json"))
CURRENT_MEASUREMENT_LENGTH=`cat /tmp/tmpdata.json | jq -r ".results[0].series | length" `
for (( j=0; j<=$(($CURRENT_MEASUREMENT_LENGTH - 1 )); j++)); do
DATAPOINTS=`cat /tmp/tmpdata.json | jq -r ".results[0].series[$j].values[0] | last"`
TAGS_VAL=""
for e in ${TAGS[*]}; do
TAGS_VAL+=`cat /tmp/tmpdata.json | jq -r ".results[0].series[$j].tags.\"$e\""`
TAGS_VAL+=";"
done
echo "$CURRENT_MEASUREMENT_NAME;$DATAPOINTS;${TAGS_VAL}$USEDTIME;" >> $DOC_NAME
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment