Skip to content

Instantly share code, notes, and snippets.

@0RIM0
Last active October 31, 2018 10:32
Show Gist options
  • Save 0RIM0/85bdd5c29495c579223bc50fd336465d to your computer and use it in GitHub Desktop.
Save 0RIM0/85bdd5c29495c579223bc50fd336465d to your computer and use it in GitHub Desktop.
s3 に stream でアップロードするときは使えない stream もある

fetch した結果を s3 にアップロード
stream を使ってアップロードしようとすると

Error: Cannot determine length of [object Object]

とエラーが出る

コード

const fetch = require("node-fetch")
const S3 = require("aws-sdk/clients/s3")

const s3 = new S3({/*接続情報*/})
const bucket = "バケット"

async function save(url, path){
	const res = await fetch(url)
	if(!res.ok) throw new Error(res.status)
	
	await upload(path, res.body)
}

async function upload(key, value){
	return s3.putObject({
		Bucket: bucket,
		Key: key,
		Body: value,
	}).promise()
}

save(/* 適当なURL */, "test/file")

Body プロパティは Readable Stream も受け取るらしい

const fs = require("fs")
upload("test/text", fs.createReadStream("/tmp/text.txt"))

は動く

const stream = require("stream")
const pt = new stream.PassThrough()

upload("test/text", pt)
fs.createReadStream("/tmp/text.txt").pipe(pt)

はダメ

length が必須らしいので content-length を使用

async function save(url, path){
	const res = await fetch(url)
	if(!res.ok) throw new Error(res.status)
	
	if(res.headers.has("content-length")){
		await upload(path, Object.assign(res.body, {length: ~~res.headers.get("content-length")}))
	}else{
		await upload(path, await res.buffer())
	}
}

無いときは自分で計算することになるので stream じゃなくて buffer に

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment