Skip to content

Instantly share code, notes, and snippets.

@caike
Last active October 17, 2022 00:27
  • Star 10 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save caike/ebccc95bd46f5fa1404d to your computer and use it in GitHub Desktop.
Node.js streams 1 vs. 2

NodeJS Streams

  • Current implementation is known as streams2.
  • Introduced in node v0.10.
  • "suck" streams instead of "spew" streams.
  • Instead of data events spewing, call read() to pull data from source.
  • When there isn't any data to consume, then read() will return undefined.
  • Adding a data event listener will switch the Readable stream into "old mode", where data is emitted as soon as it is available rather than waiting for you to call read() to consume it. This requires you to handle backpressure problems manually.
  • The pipe method helps write less code and handles back-pressure.
  • If you add an end listener and don't ever read() or pipe(), it'll never emit end.

Resources

Isaac Z. Presentation
Tim Caswell
Stream Compatibility
Getting Started With Streams2

// streams1
// works in node v0.6.21
// works in node v0.10.29 (latest stable)
// Tested with `curl -d 'akshdakjhdakjhsdkajhdjkahdsjka' http://localhost:8080`
var http = require('http');
http.createServer(function(req, res) {
console.log('Got Request');
res.writeHead(200);
req.on('data', function(chunk){
console.log('got %d bytes of data', chunk.length);
});
req.on('end', function() {
res.end('over');
});
}).listen(8080);
var canWrite = true;
readStream.on('readable', function() {
while(canWrite && (null !== (chunk = readStream.read()))){
var bufferGood = writeStream.write(chunk)
if (!bufferGood) canWrite = false;
}
})
writeStream.on('drain', function() {
canWrite = true;
});
// streams2
// works in v0.10.29
// does not work in v.0.6.21
// Example taken from http://nodejs.org/api/stream.html#stream_class_stream_readable
// Tested with `curl -d 'akshdakjhdakjhsdkajhdjkahdsjka' http://localhost:8080`
var http = require('http');
http.createServer(function(req, res) {
console.log('Got Request');
req.on('readable', function() {
var chunk = null;
while (null !== (chunk = req.read())) {
console.log('got %d bytes of data', chunk.length);
}
});
req.on('end', function() {
res.end('Done');
});
}).listen(8080);
var http = require('http');
var fs = require('fs');
http.createServer(function(req, res) {
var newFile = fs.createWriteStream('destination.jpg');
var fileBytes = req.headers['content-length'];
var uploadedBytes = 0;
req.on('readable', function() {
var chunk = null;
while(null !== (chunk = req.read())){
uploadedBytes += chunk.length;
var progress = (uploadedBytes / fileBytes) * 100;
res.write("Progress: " + parseInt(progress, 10) + "%\n");
}
});
req.on('end', function() {
res.end('uploaded!');
});
req.pipe(newFile);
}).listen(8080);
// run with `curl --upload-file <your-file.jpg> http://localhost:8080`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment