KEY=XXXXXXXXXXXX | |
HOST="https://metrics.crisidev.org" | |
mkdir -p dashboards && for dash in $(curl -k -H "Authorization: Bearer $KEY" $HOST/api/search\?query\=\& |tr ']' '\n' |cut -d "," -f 5 |grep slug |cut -d\" -f 4); do | |
curl -k -H "Authorization: Bearer $KEY" $HOST/api/dashboards/db/$dash > dashboards/$dash.json | |
done |
Well, I succeeded exporting them but I re-importing resulted in an empty dashboard (grafana 7.5.5) :/
Hey @varac any luck importing them?
Hey @varac any luck importing them?
No - I ended up exporting + reimporting manually :/
Hello,
I tried to run something on my own to see if the dashboards get imported all at once or not. Below short script worked for me. I used @magiconair 's script to export the dashboards to my system.
for FILE in *; do
cat $FILE | jq '. * {overwrite: true, dashboard: {id: null}}' | curl -X POST
-H "Content-Type: application/json" -H "Authorization: Bearer api-key-with-write-permissions"
grafana-url/api/dashboards/db -d @- ;
done
With this all the dashboards are re-imported on grafana.
I hope it helps.
P.S - The code was taken with the help from https://github.com/monitoringartist/grafana-utils
Only is for dashboards, with datasource the command is failed:
#ERROR indicating id (basename file) in XPUT :
cat VSPP/datasources/VSPP.json.json | jq '. * {overwrite: true }' | curl -k -XPUT -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: Basic aaaaaaaa" http://localhost:3000/api/datasources/VSPP.json -d @-
result:
{"message":"Failed to update datasource. Reload new version and try again"}
#WITHOUT JQ {overwrite: true } - json datasource exported without changes:
$ cat "VSPP/datasources/VSPP.json.json" | curl -k -XPUT -H "Content-Type: application/json" -H "Accept: application/json" -H "Authorization: Basic aaaaaaaa" http://localhost:3000/api/datasources/VSPP.json -d @-
result:
{"message":"Failed to update datasource. Reload new version and try again"}
Any idea about how to import datasources ??? I try to import from export json files like (when datasource already exist - I need update the version of datasource or overwrite it )
{
"id": 39,
"orgId": 6,
"name": "VSPP",
"type": "graphite",
"typeLogoUrl": "",
"access": "proxy",
"url": "http://1.1.1.1:8066",
"password": "",
"user": "",
"database": "",
"basicAuth": true,
"basicAuthUser": "***",
"basicAuthPassword": "****",
"withCredentials": false,
"isDefault": true,
"jsonData": {
"graphiteVersion": "",
"keepCookies": []
},
"secureJsonFields": {},
"version": 5,
"readOnly": false
}
It is work! All Orgs with folders are saved. Thanks @xmsanchez for base scripts and @renatosis for 'jq'. Grafana ver 7.5.4:
#!/bin/bash
HOST='http://localhost:3000'
DASH_DIR=/path/to/dir
# Declare a list with the api keys using as a prefix the organization name plus "_" character
declare -a StringArray=("MAIN_<api-key>" "OtherOrg_<api-key>" "YetOnAnother_<api-key>")
# Iterate through api keys:
for API_KEY in "${StringArray[@]}"; do
ORG=$(echo $API_KEY | cut -d "_" -f1) # Name of the organization based on the prefix
KEY=$(echo $API_KEY | cut -d "_" -f2) # API Key for that organization after removing the prefix
# Iterate through dashboards using the current API Key
for dashboard_uid in $(curl -sS -H "Authorization: Bearer $KEY" $HOST/api/search\?query\=\& | jq -r '.[] | select( .type | contains("dash-db")) | .uid'); do
url=`echo $HOST/api/dashboards/uid/$dashboard_uid | tr -d '\r'`
dashboard_json=$(curl -sS -H "Authorization: Bearer $KEY" $url)
dashboard_title=$(echo $dashboard_json | jq -r '.dashboard | .title' | sed -r 's/[ \/]+/_/g' )
dashboard_version=$(echo $dashboard_json | jq -r '.dashboard | .version')
folder_title="$(echo $dashboard_json | jq -r '.meta | .folderTitle')"
# You can export the files like this to keep them organized by organization:
mkdir -p "$DASH_DIR/$ORG/$folder_title/dashboards_$ORG"
echo $dashboard_json | jq -r {meta:.meta}+.dashboard > $DASH_DIR/$ORG/$folder_title/dashboards_$ORG/${dashboard_title}_v${dashboard_version}.json
done
done
Solution provided by @autokilla47 works fine for exporting out of Grafana. However, if you want to export the dashboards between different servers/deployments/foundations (SB/NP,PROD) via api "api/dashboards/import", ID key needs to be "null" to allow the dashboard import. However if you run another import, even with overwrite set to "true" you get the following error
{
"message": "The dashboard has been changed by someone else",
"status": "version-mismatch"
}
Solution provided by @autokilla47 works fine for exporting out of Grafana. However, if you want to export the dashboards between different servers/deployments/foundations (SB/NP,PROD) via api "api/dashboards/import", ID key needs to be "null" to allow the dashboard import. However if you run another import, even with overwrite set to "true" you get the following error { "message": "The dashboard has been changed by someone else", "status": "version-mismatch" }
I had that issue too - instead I used the /api/dashboards/db
endpoint. I still had to set the id to null, but it seems I don't get the overwrite error you have.
See my example in python: https://github.com/Beam-Connectivity/grafana-dashboard-manager
@chan-vince I think the Version number is causing this "version-mismatch" problem i.e for a brand new dashboard, version is 1 in both SB and Non-Prod foundations after the initial successful export. Then, I went ahead and added multiple changes and saved dashboard after each change, now version in SB is 4, but never exported versions 2 & 3 to Non-Prod. When i export the dashboard from SB, even though i am resetting the ID to null with overwrite true. it still runs into this problem "version-mismatch" because i am trying to overwrite version 1 with 4, it appears to want the exact version number already deployed in Non-prod to increment it with 1.
This is awesome. The only addition I had was adding or removing the -k flag to curl depending if the Grafana instance was http or https. Thansk for sharing.
Used @chipironcin to create dashboards in folders
KEY="my_key_here"
HOST="https://my_instance.com"
GRAFANA_DASHBOARDS_FOLDER=".grafana/dashboards"
echo "Exporting Grafana dashboards from $HOST"
mkdir -p "${GRAFANA_DASHBOARDS_FOLDER}"
for dash in $(curl -sSL -k -H "Authorization: Bearer ${KEY}" "${HOST}/api/search?query=&" | jq -r '.[] | select(.type == "dash-db") | .uid'); do
curl -sSL -k -H "Authorization: Bearer ${KEY}" "${HOST}/api/dashboards/uid/$dash" | jq -r . >${GRAFANA_DASHBOARDS_FOLDER}/${dash}.json
slug=$(cat ${GRAFANA_DASHBOARDS_FOLDER}/${dash}.json | jq -r '.meta.slug')
folder=$(cat ${GRAFANA_DASHBOARDS_FOLDER}/${dash}.json | jq -r '.meta.folderTitle')
mkdir -p "${GRAFANA_DASHBOARDS_FOLDER}/${folder}"
mv ${GRAFANA_DASHBOARDS_FOLDER}/${dash}.json "${GRAFANA_DASHBOARDS_FOLDER}/${folder}/${slug}.json"
done
I used this to export dashboards and datasources
#!/usr/bin/env bash
# SOURCE VARIABLES
export token=xxxxx#admin token
export grafanaurl=http://xxx.xxx.xxx.xxx:3000/api
rm -rf backups
mkdir -p backups
cd backups
datasources=$(curl -s -H "Authorization: Bearer $token" -X GET $grafanaurl/datasources)
for uid in $(echo $datasources | jq -r '.[] | .uid'); do
uid="${uid/$'\r'/}" # remove the trailing '/r'
curl -s -H "Authorization: Bearer $token" -X GET "$grafanaurl/datasources/uid/$uid" | jq > grafana-datasource-$uid.json
slug=$(cat grafana-datasource-$uid.json | jq -r '.name')
mv grafana-datasource-$uid.json grafana-datasource-$uid-$slug.json # rename with datasource name and id
echo "Datasource $uid exported"
done
dashboards=$(curl -s -H "Authorization: Bearer $token" -X GET $grafanaurl/search?folderIds=0&query=&starred=false)
for uid in $(echo $dashboards | jq -r '.[] | .uid'); do
uid="${uid/$'\r'/}" # remove the trailing '/r'
curl -s -H "Authorization: Bearer $token" -X GET "$grafanaurl/dashboards/uid/$uid" | jq > grafana-dashboard-$uid.json
slug=$(cat grafana-dashboard-$uid.json | jq -r '.meta.slug')
mv grafana-dashboard-$uid.json grafana-dashboard-$uid-$slug.json # rename with dashboard name and id
echo "Dashboard $uid exported"
done
# https://www.ifconfig.it/hugo/2021/12/backup-grafana-dashboards-with-api-and-jq/
# https://gist.github.com/crisidev/bd52bdcc7f029be2f295
And i used this to re-import all dashboards and datasources on my second server.
Obviously, datasources are exported without passwords, and although the script below will re-import them, you have to manually re-set the datasource passwords.
#!/usr/bin/env bash
# DESTINATION VARIABLES
export token=xxxxxx #admin token
export grafanaurl=http://xxx.xxx.xxx.xxx:3000/api
cd backups
for FILE in *dashboard*; do
echo Importing Dashboard: $FILE
cat $FILE | jq '. * {overwrite: true, dashboard: {id: null}}' | curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $token" "$grafanaurl/dashboards/db" -d @- ;
done
for FILE in *datasource*; do
echo Importing Dashboard: $FILE
cat $FILE | curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer $token" "$grafanaurl/datasources/" -d @- ;
done
# https://www.ifconfig.it/hugo/2021/12/backup-grafana-dashboards-with-api-and-jq/
# https://gist.github.com/crisidev/bd52bdcc7f029be2f295
We also needed to export Grafana dashboards, tried the code posted by @KMurphs (thanks) but it only exported some but not all of the dashboards, so changed the code a bit.
#!/usr/bin/env bash
#SOURCE VARIABLES
export token=xxxxx#admin token
export grafanaurl=http://xxx.xxx.xxx.xxx:3000/api
#rm -rf backup
BACKUPDIR=backup_$(date +%Y%m%d_%H%M)
mkdir -p $BACKUPDIR
cd $BACKUPDIR
datasources=$(curl -s -H "Authorization: Bearer $token" -X GET $grafanaurl/datasources)
for uid in $(echo $datasources | jq -r '.[] | .uid'); do
uid="${uid/$'\r'/}" # remove the trailing '/r'
curl -s -H "Authorization: Bearer $token" -X GET "$grafanaurl/datasources/uid/$uid" | jq > grafana-datasource-$uid.json
slug=$(cat grafana-datasource-$uid.json | jq -r '.name')
mv grafana-datasource-$uid.json grafana-datasource-$uid-$slug.json # rename with datasource name and id
echo "Datasource $uid exported"
done
#dashboards=$(curl -s -H "Authorization: Bearer $token" -X GET $grafanaurl/search?folderIds=0&query=&starred=false)
dashboards=$(curl -s -H "Authorization: Bearer $token" -X GET $grafanaurl/search)
for uid in $(echo $dashboards | jq -r '.[] | .uid'); do
uid="${uid/$'\r'/}" # remove the trailing '/r'
curl -s -H "Authorization: Bearer $token" -X GET "$grafanaurl/dashboards/uid/$uid" | jq > grafana-dashboard-$uid.json
slug=$(cat grafana-dashboard-$uid.json | jq -r '.meta.slug')
mv grafana-dashboard-$uid.json grafana-dashboard-$uid-$slug.json # rename with dashboard name and id
echo "Dashboard $uid exported"
done
cd ..
# https://www.ifconfig.it/hugo/2021/12/backup-grafana-dashboards-with-api-and-jq/
# https://gist.github.com/crisidev/bd52bdcc7f029be2f295```
Yet another iteration with folders structure, and token auth
usage ./export.sh https://grafana.local MY_API_TOKEN
#!/bin/bash
set -o errexit
set -o pipefail
set -x
FULLURL="$1"
headers="Authorization: Bearer $2"
in_path=dashboards_raw
set -o nounset
echo "Exporting Grafana dashboards from $FULLURL"
mkdir -p $in_path
for dash in $(curl -H "$headers" -s "$FULLURL/api/search?query=&" | jq -r '.[] | select(.type == "dash-db") | .uid'); do
dash_path="$in_path/$dash.json"
curl -H "$headers" -s "$FULLURL/api/dashboards/uid/$dash" | jq -r . > $dash_path
slug=$(cat $dash_path | jq -r '.meta.slug')
folder="$(cat $dash_path | jq -r '.meta.folderTitle')"
mkdir -p "$folder"
mv -f $dash_path $folder/${dash}-${slug}.json
done
@senz
Dashboard folders can have spaces, so replace second to last line with
mv -f $dash_path "$folder/${dash}-${slug}.json"
Quick Question @senz
How do you import with the folder structure?
Hi all. In order to make the exported dashboards "importable" I had to remove the "dashboard" object that was including the dashboard itself. The json included a meta and a dashboard objects. Please change the 2nd jq to the example below to make it work:
FROM: jq -r . > dashboards/${dash}.json
TO: jq -r {meta:.meta}+.dashboard > dashboards/${dash}.json
I hope it helps!!!
This did help my situation on Grafana 8.3.3 to 8.5.2
@merlian2 did you got the solution to import dashboards in folder structure ??
Also needed to export a bunch of Dashboards adjusted the code of @senz a bit to get the proper dashboard names as the filename and to get rid of the dashboards_raw folder.
#!/bin/bash
set -o errexit
set -o pipefail
# set -x
FULLURL="$1"
headers="Authorization: Bearer $2"
in_path=dashboards_raw
set -o nounset
echo "Exporting Grafana dashboards from $FULLURL"
mkdir -p $in_path
for dash in $(curl -H "$headers" -s "$FULLURL/api/search?query=&" | jq -r '.[] | select(.type == "dash-db") | .uid'); do
curl -H "$headers" -s "$FULLURL/api/search?query=&" 1>/dev/null
dash_path="$in_path/$dash.json"
curl -H "$headers" -s "$FULLURL/api/dashboards/uid/$dash" | jq -r . > $dash_path
jq -r .dashboard $dash_path > $in_path/dashboard.json
title=$(jq -r .dashboard.title $dash_path)
folder="$(jq -r '.meta.folderTitle' $dash_path)"
mkdir -p "$folder"
mv -f $in_path/dashboard.json "$folder/${title}.json"
echo "exported $folder/${title}.json"
done
rm -r $in_path
@merlian2 @mintuguptha To import with folder structure, I make use of dashboard provisioning. See Grafana Docs
mkdir -p "$folder"
@GreenieT87 I had sub folders, so I made the following diff to your script.
Feel free to edit your post if you wish.
21c21,26
< mkdir -p "$folder"
---
> if [[ $title =~ / ]]
> then
> mkdir -p "$folder/${title%/*}"
> else
> mkdir -p "$folder"
> fi
Hi all, I want to know is it possible to export all the dashboards by only using the organizational account with username & password with Editor permissions ?
Is it even possible because I have only seen examples using either admin accounts or API Token.
Anybody have an idea what can be done in this case if there is no API Key available?
@tedraymond I have found the exact solution which I was looking for here: https://github.com/Beam-Connectivity/grafana-dashboard-manager
. It works without api token and even an account with Viewer/Editor role.
Avoid moving files
#!/bin/bash
set -o errexit
set -o pipefail
set -x
FULLURL="$1"
headers="Authorization: Bearer $2"
in_path="$3"
set -o nounset
echo "Exporting Grafana dashboards from $FULLURL"
mkdir -p $in_path
cd $in_path
for dash in $(curl -H "$headers" -s "$FULLURL/api/search?query=&" | jq -r '.[] | select(.type == "dash-db") | .uid'); do
dash_data=$(curl -H "$headers" "$FULLURL/api/dashboards/uid/$dash" )
slug=$(printf '%s' $dash_data | jq -r '.meta.slug')
folder="$(printf '%s' $dash_data | jq -r '.meta.folderTitle')"
mkdir -p "$folder"
printf '%s' $dash_data | jq -r . > $folder/${dash}-${slug}.json
done
Thanks autokilla47. Solution works still in v9.1.7.
PS. I've used API Keys, not sure if it works with new Grafana Service Accounts
I have created my API keys and a few of my systems keep giving me 'invalid authorization code'
It's the same script that runs on all the other systems without problems.
Even when I just run it manually and export the KEY it fails with this error
export KEY='eyJrIjoiSnp6N0N2WGJwZFJoYTB0OVpORW5uaGFaamwzWHo5WjQiLCJuIjoiYmFja3VwcyIsImlkIjoxfQ=='
aamlxgmp003 mckenns /home/mckenns $ echo $KEY
eyJrIjoiSnp6N0N2WGJwZFJoYTB0OVpORW5uaGFaamwzWHo5WjQiLCJuIjoiYmFja3VwcyIsImlkIjoxfQ==
aamlxgmp003 mckenns /home/mckenns $ echo $HOST
http://localhost:3000
aamlxgmp003 mckenns /home/mckenns $ curl -sS -H "Authorization: Bearer ${KEY}" $HOST/api/search?query=& | jq -r '.[] '
invalid API key
00000000000000000000000000000000
#Use bash binary to run script: bash grafana-dashboard-exporter.sh
#Generate unique API key for individual organization with admin rights with no expiry date.
#Change organization in grafana to generate API key
#For exporting dashboards and data sources only
#Grafana 8.5
#!/usr/bin/env bash
ORGS=(
"1ORGName:KEY"
"2ndORGName:KEY"
"3rdORGName:KEY")
HOST="http://0.0.0.0:3000"
#query start date
export YDATE=`date +%Y`
export MDATE=`date +%b`
FILE_DIR=/Backup/$YDATE/$MDATE
mkdir -p $FILE_DIR
fetch_fields() {
echo $(curl -sSL -f -k -H "Authorization: Bearer ${1}" "${HOST}/api/${2}" | jq -r "if type==\"array\" then .[] else . end | .${3}")
}
if [ ! -d "$FILE_DIR" ] ; then
mkdir -p "$FILE_DIR"
fi
for row in "${ORGS[@]}" ; do
ORG=${row%%:*}
KEY=${row#*:}
DIR="$FILE_DIR/$ORG"
if [ ! -d "$DIR" ] ; then
mkdir -p "$DIR"
fi
if [ ! -d "$DIR/dashboards" ] ; then
mkdir -p "$DIR/dashboards"
fi
if [ ! -d "$DIR/datasources" ] ; then
mkdir -p "$DIR/datasources"
fi
for dash in $(fetch_fields $KEY 'search?type=dash-db&query=&' 'uid'); do
uid=$(echo ${dash})
DB=$(echo $(fetch_fields $KEY "dashboards/uid/${uid}") | jq -r "if type==\"array\" then .[] else . end | .meta.slug").json
echo $DB
echo $uid
curl -f -k -H "Authorization: Bearer ${KEY}" "${HOST}/api/dashboards/uid/${uid}" > "$DIR/dashboards/$DB"
done
for id in $(fetch_fields $KEY 'datasources' 'id'); do
DS=$(echo $(fetch_fields $KEY "datasources/${id}" 'name')|sed 's/ /-/g').json
echo $DS
curl -f -k -H "Authorization: Bearer ${KEY}" "${HOST}/api/datasources/${id}" | jq '.id = null' | jq '.orgId = null' > "$DIR/datasources/$DS"
done
done
Hello guys!
I created this script https://gist.github.com/aeciopires/e1adfd808097ee9260a419263ccae099 based in work of @KMurphs
in this comment: https://gist.github.com/crisidev/bd52bdcc7f029be2f295?permalink_comment_id=4197302#gistcomment-4197302
The script can export datasources and dashboards of Grafana, but I only tested with Grafana 6.6.x
Example of execution script:
chmod +x exportGrafanaDashboadsAndDatasources.sh
./exportGrafanaDashboadsAndDatasources.sh
[INFO] Datasource AWS_Cloudwatch_PRD-125 exported
[INFO] Encrypting the file ./backup_grafana_dashboards_20230321_1347/datasources/grafana-datasource-AWS_Cloudwatch_PRD-125.json...
[..]
[INFO] Dashboard alerts-prometheus-kGqyea3Zz exported
[..]
Yes. I just successfully exported all my dashboards with the latest script from https://gist.github.com/crisidev/bd52bdcc7f029be2f295#gistcomment-3593795, and I have different dashboard folders.