Skip to content

Instantly share code, notes, and snippets.

@PaulMougel
Last active May 1, 2024 12:38
Show Gist options
  • Star 56 You must be signed in to star a gist
  • Fork 17 You must be signed in to fork a gist
  • Save PaulMougel/7511372 to your computer and use it in GitHub Desktop.
Save PaulMougel/7511372 to your computer and use it in GitHub Desktop.
File upload in Node.js to an Express server, using streams
// node: v0.10.21
// request: 2.27.0
var request = require('request');
var fs = require('fs');
var r = request.post("http://server.com:3000/");
// See http://nodejs.org/api/stream.html#stream_new_stream_readable_options
// for more information about the highWaterMark
// Basically, this will make the stream emit smaller chunks of data (ie. more precise upload state)
var upload = fs.createReadStream('f.jpg', { highWaterMark: 500 });
upload.pipe(r);
var upload_progress = 0;
upload.on("data", function (chunk) {
upload_progress += chunk.length
console.log(new Date(), upload_progress);
})
upload.on("end", function (res) {
console.log('Finished');
})
// node: v0.10.7
// express: 3.4.4
var fs = require('fs');
var express = require('express');
var app = express();
app.post('/', function (req, res, next) {
req.pipe(fs.createWriteStream('./uploadFile'));
req.on('end', next);
});
app.listen(3000);
@PaulMougel
Copy link
Author

$ node client.js
Sun Nov 17 2013 12:42:27 GMT+0100 (CET) 65536
Sun Nov 17 2013 12:42:27 GMT+0100 (CET) 131072
Sun Nov 17 2013 12:42:27 GMT+0100 (CET) 196608
Sun Nov 17 2013 12:42:32 GMT+0100 (CET) 262144
Sun Nov 17 2013 12:42:36 GMT+0100 (CET) 327680
Sun Nov 17 2013 12:42:40 GMT+0100 (CET) 393216
Sun Nov 17 2013 12:42:44 GMT+0100 (CET) 458752
Sun Nov 17 2013 12:42:48 GMT+0100 (CET) 524288
Sun Nov 17 2013 12:42:52 GMT+0100 (CET) 589824
Sun Nov 17 2013 12:42:56 GMT+0100 (CET) 655360
Sun Nov 17 2013 12:43:04 GMT+0100 (CET) 720896
Sun Nov 17 2013 12:43:06 GMT+0100 (CET) 786432
Sun Nov 17 2013 12:43:11 GMT+0100 (CET) 809814
Finished

(without the highWatermark parameter on the file stream, which just spams the console log)

@Zikoel
Copy link

Zikoel commented Nov 3, 2017

Hi, I try it and its work! Very nice... but what if I want receive a streamed big file from a ejs page?

@wrz-wrs
Copy link

wrz-wrs commented Dec 25, 2017

thanks

@saurabhshr123
Copy link

I need to receive video file from post request then need to upload on youtube server. This code is not working in this case. kindly suggest thanks.

@azarus
Copy link

azarus commented Feb 11, 2019

options.headers = {
            "Content-Type": "application/octet-stream",
            "Content-Disposition": "attachment"
        };

I would recommend to set the Content-Type in request. Because using express body parsers will stop the data stream and the upload will hang if the streamed data contains json like structures or urls.

@clewertonx1
Copy link

THANKS <3

@PandianSF
Copy link

hi I have one doubt how to save the data in the file in express.js

@dosipyanCedrus
Copy link

dosipyanCedrus commented Apr 1, 2024

How to to open uploaded file?
The req.on('end', next) means that piping is finished.
What if it is needed to open the file when it is uploaded?

UPD

var upload fs.createWriteStream('./uploadFile')
await req.pipe(upload);
upload.on('close', () => {
     // upload is ready
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment