Skip to content

Instantly share code, notes, and snippets.

@emtudo
Last active August 2, 2022 20:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save emtudo/f0e0771a4b07ce65dd9cffc5938b568c to your computer and use it in GitHub Desktop.
Save emtudo/f0e0771a4b07ce65dd9cffc5938b568c to your computer and use it in GitHub Desktop.
read sqs and save to file
import { SQSClient, ReceiveMessageCommand } from "@aws-sdk/client-sqs";
import { join } from 'node:path'
import { fileURLToPath } from 'node:url'
import { writeFile } from 'node:fs/promises'
const QueueUrl = ""
const client = new SQSClient({ region: "sa-east-1" });
const TARGET_DIR = join(fileURLToPath(new URL('.', import.meta.url)), 'data')
const command = new ReceiveMessageCommand({
QueueUrl,
MaxNumberOfMessages: 10,
WaitTimeSeconds: 5,
});
while (true) {
const { Messages } = await client.send(command);
if (!Messages || Messages.length == 0) {
console.warn('no more messages')
break
}
console.log(`load new ${Messages.length} messages`)
const promises = Messages.map(msg => {
return writeFile(join(TARGET_DIR, `${msg.MessageId}.json`), JSON.stringify(msg, null, 2))
})
await Promise.allSettled(promises)
.then(results => {
const errors = results.filter(row => row.status === 'rejected')
if (errors.length > 0) {
errors.forEach(err => {
console.error(err)
})
}
})
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment