How to let users upload images to S3 with Biff

howto.md

How to let users upload images to S3 with Biff

UPDATE: Biff now includes a com.biffweb/s3-request function that can be used with the examples below, with some slight modifications.

Add the provided s3.clj file to your project, and set the following keys in config.edn and secrets.env:

;; config.edn
:s3/origin "https://nyc3.digitaloceanspaces.com"
:s3/edge "https://example.nyc3.cdn.digitaloceanspaces.com"
:s3/bucket "your-bucket"
:s3/access-key "..."
:s3/secret-key "S3_SECRET_KEY"

# secrets.env
S3_SECRET_KEY=...

S3 examples

Upload a publicly accessible object:

(s3/request ctx
            {:bucket "your-bucket"
             :method "PUT"
             :key "some-key"
             :body "some-body"
             :headers {"x-amz-acl" "public-read"
                       "content-type" "text/plain"}})

• :bucket is optional; if unset, (:s3/bucket ctx) will be used.
• :key will be coerced to a string.
• :body can be a string or a file.
• The public url will be (str (:s3/edge ctx) "/some-key").
• To make the object private, set "x-amz-acl" to "private".

Retrieve an object (mainly useful for private objects):

(:body (s3/request ctx
                   {:bucket "your-bucket"
                    :method "GET"
                    :key "..."}))

Upload an image from a file input

Add a file input to your UI. It will trigger a request as soon as the user selects a file. Don't forget :hx-encoding:

[:input {:type "file"
         :name "image"
         :accept "image/apng, image/avif, image/gif, image/jpeg, image/png, image/svg+xml, image/webp"
         :hx-encoding "multipart/form-data"
         :hx-post "/upload"
         :hx-swap "outerHTML"}]

(You may want to use hx-indicator while the file uploads.)

Add a backend route that receives the image and uploads it to S3:

(defn upload-image [{:keys [s3/edge multipart-params] :as ctx}]
  (let [image-id (random-uuid)
        {:keys [tempfile content-type]} (get multipart-params "image")
        url (str edge "/" image-id)]
    (s3/request ctx {:method "PUT"
                     :key image-id
                     :body tempfile
                     :headers {"x-amz-acl" "public-read"
                               "content-type" content-type}})
    [:img {:src url}]))

(def plugin
  {:routes [["/upload" {:post upload-image}]]})

After the upload completes, the input element will be replaced with an img element.

Note: It is possible for the backend to give a signed URL to the client which allows them to upload the image directly to S3. I haven't bothered figuring out how to do that.

s3.clj

(ns com.example.s3
  (:require [com.biffweb :as biff]
            [buddy.core.mac :as mac]
            [clj-http.client :as http]
            [clojure.string :as str]))

(defn hmac-sha1-base64 [secret s]
  (-> (mac/hash s {:key secret :alg :hmac+sha1})
      biff/base64-encode))

(defn md5-base64 [body]
  (with-open [f (cond
                 (string? body) (java.io.ByteArrayInputStream. (.getBytes body))
                 :else (java.io.FileInputStream. body))]
    (let [buffer (byte-array 1024)
          md (java.security.MessageDigest/getInstance "MD5")]
      (loop [nread (.read f buffer)]
        (if (pos? nread)
          (do (.update md buffer 0 nread)
              (recur (.read f buffer)))
          (biff/base64-encode (.digest md)))))))

(defn format-date [date & [format]]
  (.format (doto (new java.text.SimpleDateFormat (or format biff/rfc3339))
             (.setTimeZone (java.util.TimeZone/getTimeZone "UTC")))
           date))

(defn body->bytes [body]
  (cond
   (string? body) (.getBytes body)
   :else (let [out (byte-array (.length body))]
           (with-open [in (java.io.FileInputStream. body)]
             (.read in out)
             out))))

(defn request [{:keys [biff/secret
                       s3/origin
                       s3/access-key]
                default-bucket :s3/bucket}
               {:keys [method
                       key
                       body
                       headers
                       bucket]}]
  ;; See https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html
  (let [bucket (or bucket default-bucket)
        date (format-date (java.util.Date.) "EEE, dd MMM yyyy HH:mm:ss Z")
        path (str "/" bucket "/" key)
        md5 (some-> body md5-base64)
        headers' (->> headers
                      (map (fn [[k v]]
                             [(str/trim (str/lower-case k)) (str/trim v)]))
                      (into {}))
        content-type (get headers' "content-type")
        headers' (->> headers'
                      (filter (fn [[k v]]
                                (str/starts-with? k "x-amz-")))
                      (sort-by first)
                      (map (fn [[k v]]
                             (str k ":" v "\n")))
                      (apply str))
        string-to-sign (str method "\n" md5 "\n" content-type "\n" date "\n" headers' path)
        signature (hmac-sha1-base64 (secret :s3/secret-key) string-to-sign)
        auth (str "AWS " access-key ":" signature)
        s3-opts {:method method
                 :url (str origin path)
                 :headers (merge {"Authorization" auth
                                  "Date" date
                                  "Content-MD5" md5}
                                 headers)
                 :body (some-> body body->bytes)}]
    (http/request s3-opts)))