Skip to content

Instantly share code, notes, and snippets.

@jsuryahyd
Last active June 15, 2019 11:30
Show Gist options
  • Save jsuryahyd/e1e27f4a354513a9da680fa9c4d2cb7f to your computer and use it in GitHub Desktop.
Save jsuryahyd/e1e27f4a354513a9da680fa9c4d2cb7f to your computer and use it in GitHub Desktop.
write large text to file using nodejs
//writes a billion numbers to file. size: 9.3GB
let writeStream = fs.createWriteStream("./output");
writeStream.on('open',async ()=>{
while (k<=10**9){
let written = k%10 == 0 ? writeStream.write(k.toString()+"\n") : writeStream.write(k.toString()+" ");
/**
imp stuff..., without which out-of-memory error is thrown.
[https://stackoverflow.com/a/40221132/7314900] stream.write() method returns false when stream has large enough data. so,
*/
if(!written){
//https://stackoverflow.com/a/50360972/7314900 - update-II
await new Promise(resolve => writeStream.once('drain',resolve));
}
k++;
}
writeStream.end();
);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment