Skip to content

Instantly share code, notes, and snippets.

@jakemcgraw
Created March 16, 2010 15:07
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save jakemcgraw/334069 to your computer and use it in GitHub Desktop.
Read a stream in nodejs
// use tail -f /file/to/watch | node process_stdio.js to consume a stream
var partial = "", queue = [];
stdio = process.stdio;
stdio.open();
// Grab whole lines off of stdio
stdio.addListener("data", function(chunk) {
if (chunk) {
if ("" != partial) {
chunk = partial+chunk;
partial = "";
}
var data = chunk.split("\n");
if (!chunk.match(/\n$/)) {
partial += data.pop();
}
queue = queue.concat(data.map(function(n){return n.replace(/(^\s+|\s+$)/, "");}).filter(function(n){return "" != n;}));
}
});
// Process messages on queue every 100ms (probably can bring this down)
setInterval(function(){
if (queue.length) {
var batch = queue.splice(0, 250);
}
}, 100);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment