Instantly share code, notes, and snippets.

Embed
What would you like to do?
Example of doing a multipart upload in Go (golang)
package main
import (
"bytes"
"fmt"
"io"
"log"
"mime/multipart"
"net/http"
"os"
"path/filepath"
)
// Creates a new file upload http request with optional extra params
func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile(paramName, filepath.Base(path))
if err != nil {
return nil, err
}
_, err = io.Copy(part, file)
for key, val := range params {
_ = writer.WriteField(key, val)
}
err = writer.Close()
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", uri, body)
req.Header.Set("Content-Type", writer.FormDataContentType())
return req, err
}
func main() {
path, _ := os.Getwd()
path += "/test.pdf"
extraParams := map[string]string{
"title": "My Document",
"author": "Matt Aimonetti",
"description": "A document with all the Go programming language secrets",
}
request, err := newfileUploadRequest("https://google.com/upload", extraParams, "file", "/tmp/doc.pdf")
if err != nil {
log.Fatal(err)
}
client := &http.Client{}
resp, err := client.Do(request)
if err != nil {
log.Fatal(err)
} else {
body := &bytes.Buffer{}
_, err := body.ReadFrom(resp.Body)
if err != nil {
log.Fatal(err)
}
resp.Body.Close()
fmt.Println(resp.StatusCode)
fmt.Println(resp.Header)
fmt.Println(body)
}
}
@themihai

This comment has been minimized.

themihai commented May 9, 2014

I'm wondering if it would be possible to upload a 'file' from memory (not from disk) ?

@mkaz

This comment has been minimized.

mkaz commented May 30, 2014

Thanks for the helpful gist, I think the only example I saw with a file upload and passing parameters. One thing, you should add a Content-Type header to the request.

For example, the return of newfileUploadRequest() :

request := http.NewRequest("POST", uri, body)
request.Header.Add("Content-Type", writer.FormDataContentType())
return request
@wzhliang

This comment has been minimized.

wzhliang commented Sep 25, 2014

@mkaz that addition save a lot of my time!

@quxiao

This comment has been minimized.

quxiao commented Jan 23, 2015

thanks! @mkaz

@sklyar

This comment has been minimized.

sklyar commented Feb 20, 2015

Thanks. Do not tell me how to send multiple files at once?

@ystreibel

This comment has been minimized.

ystreibel commented Oct 6, 2015

👍 @mkaz thanks a lot

request, err := http.NewRequest("POST", uri, body)
request.Header.Add("Content-Type", writer.FormDataContentType())
return request, err
@xiconet

This comment has been minimized.

xiconet commented Oct 24, 2015

Works, but memory usage is still high (at least in my hands), even using io.copy()

@filewalkwithme

This comment has been minimized.

filewalkwithme commented Nov 24, 2015

@mkaz, thanks, bro!!

@ZetaChow

This comment has been minimized.

ZetaChow commented Jan 16, 2016

@mkaz thanks!!

@kreshikhin

This comment has been minimized.

kreshikhin commented Feb 12, 2016

@mkaz thanks, you'd saved my day -))

@bigwheel

This comment has been minimized.

bigwheel commented Mar 3, 2016

@mkaz ty!

@skyleelove

This comment has been minimized.

skyleelove commented May 10, 2016

hi, this example is client.Could you show a example for server.Thanks.

@cherrot

This comment has been minimized.

cherrot commented Nov 29, 2016

in resp.Body.Read(bodyContent), bodyContent should not be a 0-length slice. Instead, initialize it using:

var bodyContent = make([]byte, contentLengthOfThisRequest)
@ghost

This comment has been minimized.

ghost commented Jan 7, 2017

This reads the whole file to RAM which is sometimes impossible 😓

@bacongobbler

This comment has been minimized.

bacongobbler commented Jan 31, 2017

Instead of reading the entire file into memory and then writing it to the multipart form, just open the file, defer file.Close() then call io.Copy. For example:

func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
	file, err := os.Open(path)
	if err != nil {
		return nil, err
	}
	defer file.Close()

	fi, err := file.Stat()
	if err != nil {
		return nil, err
	}

	body := new(bytes.Buffer)
	writer := multipart.NewWriter(body)
	part, err := writer.CreateFormFile(paramName, fi.Name())
	if err != nil {
		return nil, err
	}
	io.Copy(part, file)

	...

This requires adding io to the package import list.

@nullne

This comment has been minimized.

nullne commented Feb 4, 2017

@bacongobbler. thanks

@sclevine

This comment has been minimized.

sclevine commented Mar 25, 2017

@bacongobbler @nullne @mattetti If you're interested in a streaming solution that doesn't require reading the entire file into memory at all (ex. for large files, or for production use in gateways), see here:
https://github.com/sclevine/cflocal/blob/49495238fad2959061bef7a23c6b28da8734f838/remote/droplet.go#L21-L58

(Worth noting: S3 doesn't accept chunked multi-part uploads, so you have to calculate the length ahead of time if that's where you're sending the file.)

@vicbaily528

This comment has been minimized.

vicbaily528 commented Nov 7, 2017

I found a problem:
When I simulated uploading a file in golang to the spring restful API, I found that the uploaded file was incorrect when I uploaded it using your method. Although it can get the data.
@PostMapping("/demo") public String postdemo(@RequestParam("userName") String userName, @RequestBody byte[] file) throws IOException { System.out.println(userName); if (file != null) { System.out.println(file.length); String jsonFilePath = "/local/code/12312312.zip"; File jsonFile = new File(jsonFilePath); FileOutputStream outputStream = FileUtils.openOutputStream(jsonFile); outputStream.write(file); outputStream.flush(); outputStream.close(); } return "you send post parame is " + userName; }

The header of the generated file will contain the following information

--45e6a34acb7e1b27165a75bf5052f2f4f00fc96dd04899d1bf66bd782fa1 Content-Disposition: form-data; name="file"; filename="pom.zip" Content-Type: application/octet-stream ����

If I use this code "request.Header.Add("Content-Type", writer.FormDataContentType())", then he will prompt me to get the parameters: username

@dezza

This comment has been minimized.

dezza commented Dec 21, 2017

and then just test it:

	rr := httptest.NewRecorder()
	handler := http.HandlerFunc(API.Create)
	handler.ServeHTTP(rr, req)

	if status := rr.Code; status != http.StatusOK {
		t.Errorf("handler returned wrong status code: got %v want %v",
			status, http.StatusOK)
	}
@eoinahern

This comment has been minimized.

eoinahern commented Jan 16, 2018

any examples of posting a file along with a serialized struct?

@lusi1990

This comment has been minimized.

lusi1990 commented Jan 17, 2018

@mkaz save my day

@naughtymonkey1010

This comment has been minimized.

naughtymonkey1010 commented Feb 28, 2018

@mkaz thanks

@znoodl

This comment has been minimized.

znoodl commented Mar 26, 2018

using io.pipe will be better?

@shuaihanhungry

This comment has been minimized.

shuaihanhungry commented Apr 25, 2018

@e7

This comment has been minimized.

e7 commented Jul 12, 2018

@themihai I'm wondering it too, do you have solution?

@tcr-ableton

This comment has been minimized.

tcr-ableton commented Sep 4, 2018

My two cents: using file.Stat is an triggers an unnecessary system call. You could use filepath.Base(path) in order to obtain the filename.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment