File Upload using HTTP Request node

README.md

This flow demonstrates how to perform a file upload using the HTTP Request node.

At the time of writing (February 2019), the core HTTP Request node does not yet provide a simple way to do file uploads. It's on the todo list, but we're not there yet.

This example flow consists of two parts:

• an HTTP In flow listening for POST requests to /files. This will receive the file upload and display its contents to the Debug sidebar

• an Inject driven flow that reads the file /tmp/file.txt, generates the proper msg.payload and passes that to an HTTP Request node to upload to the HTTP In flow above.

The key part of this example flow is the Function node that constructs the necessary payload. A File Upload request consists of a multipart/form-data request. This can contain multiple pieces of form data, some of which could be files.

msg.headers = {
    "Content-Type": "multipart/form-data; boundary=------------------------d74496d66958873e"
}


msg.payload = '--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="select"\r\n'+
'\r\n'+
'true\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="print"\r\n'+
'\r\n'+
'true\r\n'+
'--------------------------d74496d66958873e\r\n'+
'Content-Disposition: form-data; name="file"; filename="'+msg.filename+'"\r\n'+
'Content-Type: application/octet-stream\r\n'+
'\r\n'+
msg.payload+'\r\n'+
'--------------------------d74496d66958873e--\r\n';


return msg;

Here you can see the Function receives msg.payload and msg.filename containing the file information. They are added into the payload.

The payload also includes two other form elements - print and select, both with a value of "true".

Some observations if you want to customise this for your own needs:

• The Content-Type header defines the boundary text used to separate the different parts of the payload
• When that boundary appears in the payload it has an extra -- added in front
• The final boundary text also has an extra -- added at the end
• All newlines must by \r\n - which means you cannot construct this in the Template node.

flow.json

[{"id":"c4f91df3.caef7","type":"http in","z":"174067df.9708b8","name":"","url":"/files","method":"post","upload":true,"swaggerDoc":"","x":100,"y":100,"wires":[["cc5a37cf.86c6d8","c14febc0.522db8"]]},{"id":"ef0aaf76.2236e","type":"http response","z":"174067df.9708b8","name":"","statusCode":"","headers":{},"x":450,"y":100,"wires":[]},{"id":"cc5a37cf.86c6d8","type":"debug","z":"174067df.9708b8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"req","targetType":"msg","x":260,"y":140,"wires":[]},{"id":"6a156025.b1c9f","type":"inject","z":"174067df.9708b8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":200,"wires":[["fd62708f.a560a"]]},{"id":"9c0b2081.b1ac6","type":"function","z":"174067df.9708b8","name":"Format the header and payload","func":"msg.headers = {\n    \"Content-Type\": \"multipart/form-data; boundary=------------------------d74496d66958873e\"\n}\n\n\nmsg.payload = '--------------------------d74496d66958873e\\r\\n'+\n'Content-Disposition: form-data; name=\"select\"\\r\\n'+\n'\\r\\n'+\n'true\\r\\n'+\n'--------------------------d74496d66958873e\\r\\n'+\n'Content-Disposition: form-data; name=\"print\"\\r\\n'+\n'\\r\\n'+\n'true\\r\\n'+\n'--------------------------d74496d66958873e\\r\\n'+\n'Content-Disposition: form-data; name=\"file\"; filename=\"'+msg.filename+'\"\\r\\n'+\n'Content-Type: application/octet-stream\\r\\n'+\n'\\r\\n'+\nmsg.payload+'\\r\\n'+\n'--------------------------d74496d66958873e--\\r\\n';\n\n\nreturn msg;","outputs":1,"noerr":0,"x":230,"y":260,"wires":[["85c748e8.2d1f88"]]},{"id":"85c748e8.2d1f88","type":"http request","z":"174067df.9708b8","name":"","method":"POST","ret":"txt","paytoqs":false,"url":"http://localhost:1880/files","tls":"","proxy":"","x":470,"y":260,"wires":[["a4810d7a.f652a"]]},{"id":"a4810d7a.f652a","type":"debug","z":"174067df.9708b8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":650,"y":260,"wires":[]},{"id":"c14febc0.522db8","type":"change","z":"174067df.9708b8","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"okay","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":280,"y":100,"wires":[["ef0aaf76.2236e"]]},{"id":"fd62708f.a560a","type":"file in","z":"174067df.9708b8","name":"","filename":"/tmp/file.txt","format":"","chunk":false,"sendError":false,"x":270,"y":200,"wires":[["9c0b2081.b1ac6"]]}]

This one works for binary data also:

const genRanHex = size => [...Array(size)].map(() => Math.floor(Math.random() * 16).toString(16)).join('');

let boundary = "------------------------"+genRanHex(16);

msg.headers = {
    "Content-Type": "multipart/form-data; boundary="+boundary
}

let data_pre = Buffer.from("--"+boundary+'\r\n'+
"Content-Disposition: form-data; name=\"file\"; filename=\""+msg.filename+"\"\r\n"+
"Content-Type: application/octet-stream\r\n"+
"\r\n", "utf-8");

let data_post = Buffer.from("\r\n"+
"--"+boundary+"--\r\n", "utf-8");

msg.payload = Buffer.concat([data_pre, msg.payload, data_post]);

return msg;

I had a lot of trouble to find out what was expected. I receive a posted binary file (7bit), which I send to a conversion-service and then send the converted file to the user. The problem was that the conversion-service thought the file was a txt-file. Tried loads of conversions and the winning solution was: make sure that payload always is a buffer and never a string. That worked.

const boundary = "------------------------d74496d66958873e"

msg.headers = {
    "Content-Type": "multipart/form-data; boundary=" + boundary
}

let data_channel_id = Buffer.from("--" + boundary + '\r\n' +
    'Content-Disposition: form-data; name="channel_id"\r\n' +
    '\r\n' +
    'mxa5hxw8b7y17bmaiisi9wzpiy\r\n');

let data_bin = Buffer.from("--" + boundary + '\r\n' +
    "Content-Disposition: form-data; name=\"file\"; filename=\"" + msg.filename + "\"\r\n" +
    "Content-Type: application/image/png\r\n" +
    "\r\n", "utf-8");

let data_post = Buffer.from("\r\n" +
    "--" + boundary + "--\r\n", "utf-8");

msg.payload = Buffer.concat([data_channel_id, data_bin, msg.payload, data_post]);

return msg;

Work with binaries and one other value