Skip to content

Instantly share code, notes, and snippets.

@donmccurdy
Created October 31, 2018 20:45
Show Gist options
  • Star 6 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save donmccurdy/6cbcd8cee74301f92b4400b376efda1d to your computer and use it in GitHub Desktop.
Save donmccurdy/6cbcd8cee74301f92b4400b376efda1d to your computer and use it in GitHub Desktop.
example CSV transformation in Node.js
const fs = require('fs')
const csv = require('csv');
fs.createReadStream('data.csv')
.pipe(csv.parse({columns: true}))
.pipe(csv.transform((input) => {
return Object.assign({}, input, {
foo: input['foo'].toUpperCase()
});
}))
.pipe(csv.stringify({header: true}))
.pipe(fs.createWriteStream('./data-processed.csv'))
.on('finish', () => {
console.log('Done 🍻 ');
});
@donmccurdy
Copy link
Author

For example:

const fs = require('fs')
const csv = require('csv');

const rows = [];

fs.createReadStream('upload_7_17.csv')
  .pipe(csv.parse({columns: true}))
  .pipe(csv.transform((input) => {
    rows.push({...input});
  }))
  .on('finish', () => {
    console.log(rows);
  });

Note that this requires the data to fit in memory, so it may not scale as well as streaming row by row.

@atharva-bhange
Copy link

How can i do async operation in transform ?

@donmccurdy
Copy link
Author

Hi sorry, I'm not involved with the csv package and can't provide support for it. This is just an example I've posted as a reference. Stack Overflow or the csv github repository may be better places to ask for support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment