Instantly share code, notes, and snippets.

Embed
What would you like to do?
Example of doing a multipart upload in Go (golang)
package main
import (
"bytes"
"fmt"
"io"
"log"
"mime/multipart"
"net/http"
"os"
"path/filepath"
)
// Creates a new file upload http request with optional extra params
func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile(paramName, filepath.Base(path))
if err != nil {
return nil, err
}
_, err = io.Copy(part, file)
for key, val := range params {
_ = writer.WriteField(key, val)
}
err = writer.Close()
if err != nil {
return nil, err
}
req, err := http.NewRequest("POST", uri, body)
req.Header.Set("Content-Type", writer.FormDataContentType())
return req, err
}
func main() {
path, _ := os.Getwd()
path += "/test.pdf"
extraParams := map[string]string{
"title": "My Document",
"author": "Matt Aimonetti",
"description": "A document with all the Go programming language secrets",
}
request, err := newfileUploadRequest("https://google.com/upload", extraParams, "file", "/tmp/doc.pdf")
if err != nil {
log.Fatal(err)
}
client := &http.Client{}
resp, err := client.Do(request)
if err != nil {
log.Fatal(err)
} else {
body := &bytes.Buffer{}
_, err := body.ReadFrom(resp.Body)
if err != nil {
log.Fatal(err)
}
resp.Body.Close()
fmt.Println(resp.StatusCode)
fmt.Println(resp.Header)
fmt.Println(body)
}
}
@themihai

This comment has been minimized.

Copy link

themihai commented May 9, 2014

I'm wondering if it would be possible to upload a 'file' from memory (not from disk) ?

@mkaz

This comment has been minimized.

Copy link

mkaz commented May 30, 2014

Thanks for the helpful gist, I think the only example I saw with a file upload and passing parameters. One thing, you should add a Content-Type header to the request.

For example, the return of newfileUploadRequest() :

request := http.NewRequest("POST", uri, body)
request.Header.Add("Content-Type", writer.FormDataContentType())
return request
@wzhliang

This comment has been minimized.

Copy link

wzhliang commented Sep 25, 2014

@mkaz that addition save a lot of my time!

@quxiao

This comment has been minimized.

Copy link

quxiao commented Jan 23, 2015

thanks! @mkaz

@sklyar

This comment has been minimized.

Copy link

sklyar commented Feb 20, 2015

Thanks. Do not tell me how to send multiple files at once?

@ystreibel

This comment has been minimized.

Copy link

ystreibel commented Oct 6, 2015

👍 @mkaz thanks a lot

request, err := http.NewRequest("POST", uri, body)
request.Header.Add("Content-Type", writer.FormDataContentType())
return request, err
@xiconet

This comment has been minimized.

Copy link

xiconet commented Oct 24, 2015

Works, but memory usage is still high (at least in my hands), even using io.copy()

@filewalkwithme

This comment has been minimized.

Copy link

filewalkwithme commented Nov 24, 2015

@mkaz, thanks, bro!!

@ZetaChow

This comment has been minimized.

Copy link

ZetaChow commented Jan 16, 2016

@mkaz thanks!!

@kreshikhin

This comment has been minimized.

Copy link

kreshikhin commented Feb 12, 2016

@mkaz thanks, you'd saved my day -))

@bigwheel

This comment has been minimized.

Copy link

bigwheel commented Mar 3, 2016

@mkaz ty!

@skyleelove

This comment has been minimized.

Copy link

skyleelove commented May 10, 2016

hi, this example is client.Could you show a example for server.Thanks.

@cherrot

This comment has been minimized.

Copy link

cherrot commented Nov 29, 2016

in resp.Body.Read(bodyContent), bodyContent should not be a 0-length slice. Instead, initialize it using:

var bodyContent = make([]byte, contentLengthOfThisRequest)
@ghost

This comment has been minimized.

Copy link

ghost commented Jan 7, 2017

This reads the whole file to RAM which is sometimes impossible 😓

@bacongobbler

This comment has been minimized.

Copy link

bacongobbler commented Jan 31, 2017

Instead of reading the entire file into memory and then writing it to the multipart form, just open the file, defer file.Close() then call io.Copy. For example:

func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
	file, err := os.Open(path)
	if err != nil {
		return nil, err
	}
	defer file.Close()

	fi, err := file.Stat()
	if err != nil {
		return nil, err
	}

	body := new(bytes.Buffer)
	writer := multipart.NewWriter(body)
	part, err := writer.CreateFormFile(paramName, fi.Name())
	if err != nil {
		return nil, err
	}
	io.Copy(part, file)

	...

This requires adding io to the package import list.

@nullne

This comment has been minimized.

Copy link

nullne commented Feb 4, 2017

@bacongobbler. thanks

@sclevine

This comment has been minimized.

Copy link

sclevine commented Mar 25, 2017

@bacongobbler @nullne @mattetti If you're interested in a streaming solution that doesn't require reading the entire file into memory at all (ex. for large files, or for production use in gateways), see here:
https://github.com/sclevine/cflocal/blob/49495238fad2959061bef7a23c6b28da8734f838/remote/droplet.go#L21-L58

(Worth noting: S3 doesn't accept chunked multi-part uploads, so you have to calculate the length ahead of time if that's where you're sending the file.)

@vicbaily528

This comment has been minimized.

Copy link

vicbaily528 commented Nov 7, 2017

I found a problem:
When I simulated uploading a file in golang to the spring restful API, I found that the uploaded file was incorrect when I uploaded it using your method. Although it can get the data.
@PostMapping("/demo") public String postdemo(@RequestParam("userName") String userName, @RequestBody byte[] file) throws IOException { System.out.println(userName); if (file != null) { System.out.println(file.length); String jsonFilePath = "/local/code/12312312.zip"; File jsonFile = new File(jsonFilePath); FileOutputStream outputStream = FileUtils.openOutputStream(jsonFile); outputStream.write(file); outputStream.flush(); outputStream.close(); } return "you send post parame is " + userName; }

The header of the generated file will contain the following information

--45e6a34acb7e1b27165a75bf5052f2f4f00fc96dd04899d1bf66bd782fa1 Content-Disposition: form-data; name="file"; filename="pom.zip" Content-Type: application/octet-stream ����

If I use this code "request.Header.Add("Content-Type", writer.FormDataContentType())", then he will prompt me to get the parameters: username

@dezza

This comment has been minimized.

Copy link

dezza commented Dec 21, 2017

and then just test it:

	rr := httptest.NewRecorder()
	handler := http.HandlerFunc(API.Create)
	handler.ServeHTTP(rr, req)

	if status := rr.Code; status != http.StatusOK {
		t.Errorf("handler returned wrong status code: got %v want %v",
			status, http.StatusOK)
	}
@eoinahern

This comment has been minimized.

Copy link

eoinahern commented Jan 16, 2018

any examples of posting a file along with a serialized struct?

@lusi1990

This comment has been minimized.

Copy link

lusi1990 commented Jan 17, 2018

@mkaz save my day

@naughtymonkey1010

This comment has been minimized.

Copy link

naughtymonkey1010 commented Feb 28, 2018

@mkaz thanks

@icefed

This comment has been minimized.

Copy link

icefed commented Mar 26, 2018

using io.pipe will be better?

@shuaihanhungry

This comment has been minimized.

Copy link

shuaihanhungry commented Apr 25, 2018

@e7

This comment has been minimized.

Copy link

e7 commented Jul 12, 2018

@themihai I'm wondering it too, do you have solution?

@tcr-ableton

This comment has been minimized.

Copy link

tcr-ableton commented Sep 4, 2018

My two cents: using file.Stat is an triggers an unnecessary system call. You could use filepath.Base(path) in order to obtain the filename.

@ramadani

This comment has been minimized.

Copy link

ramadani commented Dec 31, 2018

great solution @bacongobbler, big thanks

@melissafzhang

This comment has been minimized.

Copy link

melissafzhang commented Feb 13, 2019

I'm having similar issue as @vicbaily528 where the content type is an octet stream. Any ideas how to get around this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment