Skip to content

Instantly share code, notes, and snippets.

@mbleigh
Last active March 28, 2024 19:14
Show Gist options
  • Star 80 You must be signed in to star a gist
  • Fork 29 You must be signed in to fork a gist
  • Save mbleigh/9c8680cf319ace2f506f57380da66e7d to your computer and use it in GitHub Desktop.
Save mbleigh/9c8680cf319ace2f506f57380da66e7d to your computer and use it in GitHub Desktop.
Firebase Hosting Fetch All Files

Fetch All Files from Firebase Hosting

This script fetches all of the files from the currently deployed version of a Firebase Hosting site. You must be signed in via the Firebase CLI and have "Site Viewer" permission on the site in question to be able to properly run the script.

Running via NPX

npx https://gist.github.com/mbleigh/9c8680cf319ace2f506f57380da66e7d <site_name>

Where <site_name> is your Firebase Hosting site (e.g. my-site from my-site.firebaseapp.com). The files will be downloaded into a folder <site_name>_<version_id> where the version ID is the current version of your site.

#!/usr/bin/env node
const requireAuth = require('firebase-tools/lib/requireAuth');
const api = require('firebase-tools/lib/api');
const fs = require('fs-extra');
const request = require('request');
if (!process.argv[2]) {
console.error(`
ERROR: Must supply a site name. Usage:
node fetchFiles.js <site_name>`);
process.exit(1);
}
const site = process.argv[2];
async function getLatestVersionName() {
const result = await api.request('GET', `/v1beta1/sites/${site}/releases?pageSize=1`, {
auth: true,
origin: api.hostingApiOrigin,
});
const release = (result.body.releases || [])[0];
if (release) {
return release.version.name;
}
return null;
}
const LIST_PAGE_SIZE = 1000;
async function listFiles(versionName, existing = [], pageToken = null) {
const result = await api.request('GET', `/v1beta1/${versionName}/files?pageSize=${LIST_PAGE_SIZE}${pageToken ? `&pageToken=${pageToken}` : ''}`, {auth: true, origin: api.hostingApiOrigin});
result.body.files.forEach(file => existing.push(file.path));
if (result.body.nextPageToken) {
return await listFiles(versionName, existing, result.body.nextPageToken);
}
return existing;
}
const MAX_FETCHES = 100;
(async function() {
try {
await requireAuth({}, ['https://www.googleapis.com/auth/cloud-platform']);
const v = await getLatestVersionName();
const vid = v.split('/')[v.split('/').length - 1];
const toFetch = await listFiles(v);
const dirName = `${site}_${vid}`;
let fetchesOutstanding = 0;
let fetchCount = 0;
function fetch() {
if (fetchesOutstanding >= MAX_FETCHES) {
return;
} else if (toFetch.length === 0) {
console.log();
console.log("Complete. Fetched", fetchCount, "files.");
return;
}
const f = toFetch.shift();
console.log('Fetching', f);
fetchesOutstanding++;
fetchCount++;
fs.ensureFileSync(dirName + f);
const q = request(`https://${site}.firebaseapp.com${f}`)
const ws = fs.createWriteStream(dirName + f);
q.pipe(ws);
ws.on('finish', () => {
console.log('Fetched ', f);
fetchesOutstanding--;
fetch();
});
}
fetch();
} catch (e) {
console.error("ERROR:", e.stack);
process.exit(1);
}
})();
{
"name": "firebase-hosting-fetch-version",
"version": "0.1.0",
"bin": "./fetchFiles.js",
"dependencies": {
"firebase-tools": "^7.4.0",
"fs-extra": "^8.1.0",
"request": "^2.88.0"
}
}
@hg2355
Copy link

hg2355 commented Dec 20, 2021

after the files are download - how do you actually view the code? in the files download, I don't see any familiar code from the react-application that was my latest deployment to firebase.

@Bodrie
Copy link

Bodrie commented Jan 24, 2022

Great tool! Thank you!

@jgtaylor123
Copy link

I got the "Warning: Accessing non-existent property 'padLevels' of module exports inside circular dependency" error too.

Changed the site to exclude "firebaseapp.com" and it worked like a charm. Thanks.

@mariohenriquelopes3
Copy link

thank you! is there something similar for fetch functions?

@raviganwal
Copy link

raviganwal commented Apr 24, 2022

Using the above method I am only able to fetch the compiled version of dart files, not the actual source code. If anyone knows a workaround please help.

@mbleigh
Copy link
Author

mbleigh commented Apr 24, 2022 via email

@vojta001
Copy link

vojta001 commented May 3, 2022

Should your files contain characters unacceptable in URLs, use this path to escape them first

diff --git a/fetchFiles.js b/fetchFiles.js
index 78e59ab..3409fc2 100644
--- a/fetchFiles.js
+++ b/fetchFiles.js
@@ -37,6 +37,11 @@ async function listFiles(versionName, existing = [], pageToken = null) {
   return existing;
 }

+function escapePathForURL(path) {
+  const components = path.split('/');
+  return components.map(encodeURIComponent).join('/');
+}
+
 const MAX_FETCHES = 100;

 (async function() {
@@ -62,7 +67,7 @@ const MAX_FETCHES = 100;
       fetchesOutstanding++;
       fetchCount++;
       fs.ensureFileSync(dirName + f);
-      const q = request(`https://${site}.firebaseapp.com${f}`)
+      const q = request(`https://${site}.firebaseapp.com${escapePathForURL(f)}`)
       const ws = fs.createWriteStream(dirName + f);
       q.pipe(ws);
       ws.on('finish', () => {

@mbleigh Feel free to apply it to your Gist.

@NkosikhonaD
Copy link

worked like charm

@gitdast
Copy link

gitdast commented Feb 28, 2023

THANK YOU!

@vugar005
Copy link

I had same 404 issue,
as @jinmanpak mentioned we should use site name (not website URL).
Thanks @jinmanpak

Untitled

@rosshabe
Copy link

@mbleigh Worked for me thank you a alot.

@gaberilde
Copy link

I posted the above url fix myself for convinience
https://gist.github.com/gaberilde/4c2d55d8c6de0772ca49c8b473dff442

@Enrollmedicare
Copy link

We are so confused. We have Node.js and NPM downloaded. We are logged in to our Firebase through CLI. We requested Firebase to list projects we have access to. It gave us the Project ID. We have pasted the gist.github.com/mbleigh link above and at the end we have listed our Project ID (which does not include .firebase.app at the end). We have tried with a space between the link and the project name, without a space, with <> and without <>. We continue getting this response.
npm ERR! code ENOENT
npm ERR! syscall spawn git
npm ERR! path git
npm ERR! errno -4058
npm ERR! enoent An unknown git error occurred
npm ERR! enoent This is related to npm not being able to find a file.
npm ERR! enoent

Please help anyone?! Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment