Skip to content

Instantly share code, notes, and snippets.

@edvaldoszy
Created November 21, 2019 04:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save edvaldoszy/b02097e1c204844261bba13b6e6dcd04 to your computer and use it in GitHub Desktop.
Save edvaldoszy/b02097e1c204844261bba13b6e6dcd04 to your computer and use it in GitHub Desktop.
Faz a importação de um CSV para o banco de dados
const { Writable } = require('stream');
const { createReadStream } = require('fs');
const Split = require('stream-split');
const input = createReadStream('./votacao_candidato_munzona_2018_PR.csv', { encoding: 'latin1' });
// const [nl] = Buffer.from('\n');
// const buffer = Buffer.from('Edvaldo\nSzymonek');
// console.log(buffer[7] === nl);
class DatabaseWriter extends Writable {
constructor() {
super();
this.size = 100000;
this.lines = [];
}
insert(records) {
console.log('Inserting', records.length, 'records...');
return Promise.resolve();
}
_write(buffer, encoding, done) {
const line = buffer.toString();
const values = line.split(';');
this.lines.push(values);
if (this.lines.length < this.size) {
done();
} else {
this.insert(this.lines, done)
.then(_ => {
this.lines = [];
done();
});
}
}
end() {
this.insert(this.lines)
.then(_ => console.log('Finished!'));
}
}
input.pipe(new Split('\n'))
.pipe(new DatabaseWriter());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment