Skip to content

Instantly share code, notes, and snippets.

@pedrocarrico pedrocarrico/README.md
Last active Aug 5, 2016

Embed
What would you like to do?
pg-copy-streams readable stream not finishing and multiple end events error example

How to make it run

Be sure you have postgres up and running and change the connection string in pgcopy.js. Run test.sql to create the table.

$ npm install pg@6.0.3

With the changes introduced in https://github.com/brianc/node-pg-copy-streams/pull/44:

$ npm install pg-copy-streams@1.1.1
$ # or install v1.1.2 through github since it was not published to npmjs
$ # npm install git+ssh://git@github.com/brianc/node-pg-copy-streams#v1.1.2
$ node pgcopy.js
fileStream finished
connected 192.168.99.100
pgStream ended
fileStream ended
pgStream ended
fileStream ended

Without the changes introduced in https://github.com/brianc/node-pg-copy-streams/pull/44:

$ npm install pg-copy-streams@1.1.0
$ node pgcopy.js
fileStream finished
connected 192.168.99.100
pgStream ended
fileStream ended

I've observed that the streams are being ended multiple times and also that the read stream is not being finished with the changes introduced in https://github.com/brianc/node-pg-copy-streams/pull/44.

The changes proposed in https://github.com/brianc/node-pg-copy-streams/pull/54 fixes this issue.

$ npm install  git+ssh://git@github.com/jeromew/node-pg-copy-streams#upstream-end
$ node pgcopy.js
fileStream finished
connected 192.168.99.100
pgStream ended
fileStream ended
var pg = require('pg');
var copyFrom = require('pg-copy-streams').from;
var fs = require('fs');
var client = new pg.Client('pg://user:password@192.168.99.100:5432/db'); // Change the connection string accordingly
client.on('error', (error) => {
console.log('client errored');
console.log(error);
});
var pgStream = client.query(copyFrom(`COPY TEST from STDIN CSV HEADER`));
pgStream.on('error', (error) => {
console.log('pgStream errored');
console.log(error);
})
.on('end', () => {
console.log('pgStream ended');
client.end();
})
.on('finished', () => {
console.log('pgStream finished');
});
var fileStream = fs.createReadStream('./test.csv');
fileStream.setEncoding('utf8');
fileStream.pipe(pgStream).on('error', (error) => {
console.log('fileStream errored');
console.log(error);
})
.on('end', () => {
console.log('fileStream ended');
})
.on('finish', () => {
console.log('fileStream finished');
});
client.connect((error) => {
if (error) {
return;
}
console.log('connected', client.connectionParameters.host);
});
id name
1 hello
2 world
3 foo
4 bar
5 baz
DROP TABLE IF EXISTS "test" CASCADE;
CREATE TABLE "test" (
"id" integer NOT NULL,
"name" varchar
);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.