Skip to content

Instantly share code, notes, and snippets.

@artze
Created June 11, 2019 15:15
Show Gist options
  • Save artze/1eb7d9923223c06581a218b57b413f8b to your computer and use it in GitHub Desktop.
Save artze/1eb7d9923223c06581a218b57b413f8b to your computer and use it in GitHub Desktop.
A transform Stream example with string replace feature
const stream = require('stream');
class ReplaceStream extends stream.Transform {
constructor(searchString, replaceString) {
super();
this.searchString = searchString;
this.replaceString = replaceString;
this.tailPiece = '';
}
_transform(chunk, encoding, callback) {
const pieces = (this.tailPiece + chunk).split(this.searchString); // [1]
const lastPiece = pieces[pieces.length - 1];
const tailPieceLen = this.searchString - 1; // [2]
this.tailPiece = lastPiece.slice(-tailPieceLen);
pieces[pieces.length - 1] = lastPiece.slice(0, -tailPieceLen); // [3]
this.push(pieces.join(this.replaceString)); // [4]
callback();
}
_flush(callback) {
this.push(this.tailPiece); // [5]
callback();
}
}
const rs = new ReplaceStream('dog', 'wizard');
rs.on('data', (chunk) => {
console.log(chunk.toString());
})
rs.write('a quick brown fox jumps over the lazy dog.')
rs.write('a quick brown f')
rs.write('ox jumps over the lazy d')
rs.write('og. a quick brown fox jumps over the lazy do')
rs.write('g. a quick br');
rs.end()
// This script replaces a searchString in a text stream with a replaceString.
// The tricky part: searchString may be truncated on one chunk, and continues on
// in the next chunk. The approach is to store the last bit of a chunk in a
// variable and prepending it to the next chunk.
// [1] Prepend previous tailPiece and splits string into array with searchString
// as delimiter
// [2] The length of tailPiece only needs to be (searchString.length -1). If
// searchString happens to at the end, it would have been picked up earlier
// anyway
// [3] Remove tailPiece part before pushing into internal buffer
// [4] Push chunk into internal buffer (similar to Readable stream set up)
// [5] _flush is called at the end of stream to push any last chunk of data into
// internal buffer (to be consumed as Readable stream) before end of stream.
// In this case, the very last tailPiece is pushed, which would otherwise be
// missed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment