Skip to content

Instantly share code, notes, and snippets.

@allenmichael
Created August 30, 2023 23:24
Show Gist options
  • Save allenmichael/dceec7fdddd796721437804947b5179d to your computer and use it in GitHub Desktop.
Save allenmichael/dceec7fdddd796721437804947b5179d to your computer and use it in GitHub Desktop.
Upload in memory data direct to S3
import { randomUUID } from "crypto";
import { S3Client } from "@aws-sdk/client-s3";
import { Upload } from "@aws-sdk/lib-storage";
import { PassThrough } from "stream";
const s3Client = new S3Client({});
(async () => {
const passThrough = new PassThrough();
for (const person of ["person-1", "person-2", "person-3"]) {
const example = { Item: {} } as { Item: { id: { S: string }, person: { S: string } } }
example.Item.id = { S: randomUUID() }
example.Item.person = { S: person }
passThrough.write(`${JSON.stringify(example)}\n`);
}
passThrough.end();
try {
const result = await new Upload({
client: s3Client,
params: {
Bucket: "<bucket-name>",
Key: "test.json",
Body: passThrough
}
}).done()
console.log(result)
} catch (err) {
console.error(err);
}
})()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment