Skip to content

Instantly share code, notes, and snippets.

@spidgorny
Created January 22, 2024 21:45
Show Gist options
  • Save spidgorny/5a0d7f524ce888670947ac97a5c6e572 to your computer and use it in GitHub Desktop.
Save spidgorny/5a0d7f524ce888670947ac97a5c6e572 to your computer and use it in GitHub Desktop.
download file from s3 with await
async function waitForDownloadStreamToFinish(
readable: StreamingBlobPayloadOutputTypes,
writeStream: fs.WriteStream,
) {
return new Promise(async (resolve, reject) => {
(readable as http.IncomingMessage)
.pipe(writeStream)
.on("error", (err: Error) => reject(err))
.on("close", () => resolve(null));
});
}
let totalBytes = 0;
let progressHandler = new Transform({
transform(chunk, encoding, callback) {
totalBytes += chunk.length;
onlyOncePerSecond(() => {
logger.log("Download S3", totalBytes, "/", fileSize);
});
this.push(chunk);
callback();
},
});
const params = {
Bucket: process.env.BUCKET!,
Key: Key,
progressHandler,
};
// logger.log(params);
const res = await s3.send(new GetObjectCommand(params));
if (!fs.existsSync(path.dirname(restoreTo))) {
fs.mkdirSync(path.dirname(restoreTo), { mode: 0o777, recursive: true });
}
let writeStream = fs.createWriteStream(restoreTo);
await waitForDownloadStreamToFinish(res.Body, writeStream);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment