Skip to content

Instantly share code, notes, and snippets.

@polarnik
Last active July 10, 2023 17:07
Show Gist options
  • Save polarnik/8d3d769aeb416ff812a44e9f1cede5db to your computer and use it in GitHub Desktop.
Save polarnik/8d3d769aeb416ff812a44e9f1cede5db to your computer and use it in GitHub Desktop.
Migrate InfluxDB metrics from InfluxDB2 to VictoriaMetrics
influxd inspect export-lp --bucket-id 5d69c811b871b876 --engine-path /var/lib/influxdb2/engine --output-path /var/lib/influxdb2/ClientMetricsExport.txt
cd /var/lib/influxdb2/
split -d -l 1000 -a 10 ./ClientMetricsExport.txt 1k_lines
files=(1k_lines*)
for ((i=${#files[@]}-1; i>=0; i--)); do
echo "${files[$i]}"
curl --verbose -i -XPOST --data-binary "@${files[$i]}" 'http://victoriaMetrics:8428/write?db=ClientMetrics&rp=auto&precision=n'
done
rm 1k_lines*
rm ClientMetricsExport.txt
@polarnik
Copy link
Author

Performance metrics:

telegram-cloud-photo-size-2-5287710104587718542-y
2 requests per second

telegram-cloud-photo-size-2-5287710104587718543-y
2000 points per second

telegram-cloud-photo-size-2-5287710104587718544-y
50 Kbytes per second

I need to migrate 30 000 000 points. This action lasts about 4 hours

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment