Skip to content

Instantly share code, notes, and snippets.

@mshakhomirov
Created November 28, 2021 14:42
Show Gist options
  • Save mshakhomirov/491c4eb380e26496103d8bbb236e208c to your computer and use it in GitHub Desktop.
Save mshakhomirov/491c4eb380e26496103d8bbb236e208c to your computer and use it in GitHub Desktop.
const queryDbAndSave = async(connection, sql, totalRecords, ts, s3key, bucket, output = 'local', dryRun = false) => {
return new Promise((resolve, reject) => {
if (output === 'local') {
const outputStream = fs.createWriteStream('output.csv', { encoding: 'utf8' });
s1.stream({ highWaterMark: BATCH_SIZE }) // stream() is just to wrap into pipeable object, and not to enable 'result' events, they are emitted anyway.
.pipe(csvstringify({ header: true }))
.pipe(outputStream)
.on('finish', () => { resolve('saved data locally'); });
});
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment