Skip to content

Instantly share code, notes, and snippets.

@mrcoles
Last active March 29, 2021 02:20
Show Gist options
  • Save mrcoles/c1ddc253214f0271e07496625564aef7 to your computer and use it in GitHub Desktop.
Save mrcoles/c1ddc253214f0271e07496625564aef7 to your computer and use it in GitHub Desktop.
Snippet for doing auto-pagination of a DynamoDB scan with an async generator in NodeJS
import { DocumentClient } from "aws-sdk/lib/dynamodb/document_client";
export async function* autoPaginateScan<I extends DocumentClient.AttributeMap>(
docClient: DocumentClient,
params: DocumentClient.ScanInput
) {
while (true) {
const data = await docClient.scan(params).promise();
if (data.Items && data.Items.length) {
const items = data.Items as I[];
yield* items;
}
if (data.LastEvaluatedKey === undefined) {
break;
}
params = { ...params, ExclusiveStartKey: data.LastEvaluatedKey };
}
}
type Item = { id: string; name: string; description?: string };
const itemSeq = fullScanSeq<Item>(docClient, {
TableName: "items",
FilterExpression: "attribute_not_exists(#d)",
ExpressionAttributeNames: { "#d": "description" }
});
for await (let item of items) {
console.log(`item: id=${item.id}, name=${item.name}`);
}
@mrcoles
Copy link
Author

mrcoles commented Mar 29, 2021

These snippets are for:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment