Skip to content

Instantly share code, notes, and snippets.

@matchu
Last active August 25, 2022 21:31
Show Gist options
  • Save matchu/e00442983006857d3a4a4563f7953a6a to your computer and use it in GitHub Desktop.
Save matchu/e00442983006857d3a4a4563f7953a6a to your computer and use it in GitHub Desktop.
Download backups of all your Heroku apps
/**
* Run this in Node to download all your Heroku apps' databases as
* Postgres .dump files, in a new folder named `backups`!
*
* Automates the export process described here:
* https://devcenter.heroku.com/articles/heroku-postgres-import-export
*
* Requires the Heroku CLI, tested with v7.62.0.
*
* WARNING: NEVER run a script on your machine without understanding it!
* Please only run this once you've read and understood it in full,
* or shared it with a personal friend you trust to audit it for you.
* If you only have a handful of apps, you should follow the
* instructions in the article manually instead, as a safety
* practice.
*/
import { promisify } from "node:util";
import { execFile as plainExecFile } from "node:child_process";
const execFile = promisify(plainExecFile);
// Run `heroku apps --json` to get the names of all your apps.
const appsProcess = await execFile("heroku", ["apps", "--json"]);
const apps = JSON.parse(appsProcess.stdout);
const appNames = apps.map((app) => app.name);
// Run `heroku pg:backups:capture --app APP_NAME` on each app.
//
// We do the capture step in parallel, because it can take time and it all
// happens on separate cloud processes anyway, so we may as well speed it up!
console.info(`=== Capturing backup snapshots for ${appNames.length} apps…`);
const capturedAppNames = [];
await Promise.all(
appNames.map(async (appName) => {
try {
await execFile("heroku", ["pg:backups:capture", "--app", appName]);
} catch (error) {
// Print the error, but don't reject the promise, because we still want
// the `Promise.all` to succeed and continue! Just return instead.
console.error(
`❌ [${appName}]: Could not capture backup. ` +
`(This could be expected if it has no database.) See below:\n`,
error
);
return;
}
console.info(`✅ [${appName}]: Backup ready!`);
capturedAppNames.push(appName);
})
);
// Run `heroku pg:backups:download --app APP_NAME` on each app where the
// capture step succeeded.
//
// But we do the download step sequentially, because it'll probably be
// bottlenecked on your bandwidth if anything, and I'm more worried about
// execution clarity for things that touch the user's own machine!
console.info(`=== Downloading backups for ${capturedAppNames.length} apps…`);
let downloadedAppNames = [];
for (const appName of capturedAppNames) {
console.info(`⏬ [${appName}]: Downloading backup…`);
try {
await execFile("heroku", [
"pg:backups:download",
"--app",
appName,
"--output",
`backups/${appName}.dump`,
]);
} catch (error) {
console.error(
`❌ [${appName}]: Could not download backup. See below:\n`,
error
);
continue;
}
console.info(`✅ [${appName}]: Backup saved!`);
downloadedAppNames.push(appName);
}
console.info(`All done! ${downloadedAppNames.length} app databases backed up!`);
/**
* Run this in Node to convert all `.dump` files in the `backups` folder to
* `.sql` files, using `pg_restore`.
*
* This enables you to human-read the contents of the dump file, and confirm
* that it contains the data you expect. You can safely delete the `.sql` files
* when you're done, because the `.dump` files contain the same data.
*
* Requires `pg_restore`, which generally comes installed with PostgreSQL.
* Tested against v12.6.
*
* WARNING: NEVER run a script on your machine without understanding it!
* Please only run this once you've read and understood it in full,
* or shared it with a personal friend you trust to audit it for you.
*/
import { readdir } from "node:fs/promises";
import { promisify } from "node:util";
import { execFile as plainExecFile } from "node:child_process";
const execFile = promisify(plainExecFile);
const fileNames = await readdir("backups");
const appNames = fileNames
.filter((n) => n.endsWith(".dump"))
.map((n) => n.match(/^(.+)\.dump$/)[1]);
for (const appName of appNames) {
console.info(`📝 [${appName}]: Copying backup to .sql…`);
try {
await execFile(`pg_restore`, [
"--file",
`backups/${appName}.sql`,
`backups/${appName}.dump`,
]);
} catch (error) {
console.error(
`❌ [${appName}]: Could not copy to .sql. See below:\n`,
error
);
continue;
}
console.info(`✅ [${appName}]: Copied backup to .sql!`);
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment