Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Streaming from ps-promise to json2csv to express response
// This code queries postgresql and streams the results into a csv
// Works like a charm for a download button, no hangup for large files!
const Json2csvTransform = require('json2csv').Transform;
const QueryStream = require('pg-query-stream');
const JSONStream = require('JSONStream');
router.get('/download.csv', async (req, res) => {
const { query } = req;
const SQL = 'SELECT * FROM table';
const qs = new QueryStream(SQL);
const transformOpts = { highWaterMark: 16384, encoding: 'utf-8' };
const json2csv = new Json2csvTransform({}, transformOpts);
// Set approrpiate download headers
res.setHeader('Content-disposition', 'attachment; filename=projects.csv');
res.writeHead(200, { 'Content-Type': 'text/csv' });
// Flush the headers before we start pushing the CSV content
res.flushHeaders();
db.stream(qs, (s) => {
// initiate streaming into the console:
s.pipe(JSONStream.stringify()).pipe(json2csv).pipe(res);
})
.then((data) => {
console.log(
'Total rows processed:', data.processed,
'Duration in milliseconds:', data.duration,
);
})
.catch((error) => {
console.log('ERROR:', error);
});
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.