Skip to content

Instantly share code, notes, and snippets.

@irony

irony/encode-cluster.js

Last active April 28, 2016 18:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save irony/d44cbda0c71707e89274edd63fcc3d35 to your computer and use it in GitHub Desktop.
Save irony/d44cbda0c71707e89274edd63fcc3d35 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
encode-cluster.js
var fs = require('fs')
var Iconv = require('iconv').Iconv
var cluster = require('cluster')
if (cluster.isMaster) [1, 2, 3, 4].forEach(_ => cluster.fork())
var converter = new Iconv('UTF-8', 'UTF-16LE')
var input = fs.createReadStream('./enwik8')
var out = fs.createWriteStream('outfile' + new Date())
input.pipe(converter).pipe(out)
out.on('close', _ => process.exit())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment