Skip to content

Instantly share code, notes, and snippets.

@realgenekim
Last active March 4, 2024 20:05
Show Gist options
  • Save realgenekim/17f9a7ae48aaf2e03df3cc80326a5094 to your computer and use it in GitHub Desktop.
Save realgenekim/17f9a7ae48aaf2e03df3cc80326a5094 to your computer and use it in GitHub Desktop.
A monsterously bad function before rewriting it. This was before rewriting it, inspired by @christoph-neumann and @justone in their Functional Design in Clojure podcast!!
(>defn interpret-photo-from-client
" input: db, photo/id (uuid), prompt string (can be 'default'), and options map "
[db uuid prompt & {:keys [async? model]
:or {async? false} :as opts}]
[(? #(instance? xtdb.node.XtdbNode %)) uuid? string? (s/* (s/or :keyword keyword? :bool boolean?)) => map?]
(log/warn :prompt-photo :model model :uuid uuid :async? async?)
(let [
record (xtp/photo-xtdb-fresh-url-uuid db uuid)
url (-> record
:photo/url
vu/xform-url-size-big)
b64 (-> url
(ol/url->stream)
(ol/stream-to-base64))
prompt (if (= prompt "default")
(slurp "resources-openai/images/podcast-screenshot.txt")
prompt)
_ (log/warn :prompt-photo :prompt prompt)
_ (log/warn :prompt-photo :async? async? :uuid uuid :url url)
;summary (slurp "/tmp/summary")
begin-ms (System/currentTimeMillis)
; must reutrn map
summary (time
(if (= model :gpt-4-vision-preview)
(let [retval (gpt4v/prompt-photo b64 prompt)
_ (log/warn :prompt-photo :first-retval retval)
interpreted (gpt4v/interpret-prompt-photo {:summary (-> retval :summary)})]
interpreted)
; llava
(if async?
(ol/interpret-photo-async! b64 {:prompt prompt
:json? true})
(ol/interpret-photo-sync! b64 {:prompt prompt}))))
_ (do
(def SUMMMARY summary)
(def UUID uuid)
(def RECORD record)
(def URL url)
(ol/write-decoded-base64-to-file b64 "/tmp/decoded.jpg")
(spit "/tmp/summary" summary)
0)
elapsed-ms (- (System/currentTimeMillis) begin-ms)
;new-photo-record (create-photo-summary-record-and-attach-to-parent! db uuid summary elapsed-ms)]
retval {:url url
:summary summary}]
;(def NEWRECORD new-photo-record)
(log/warn :prompt-photo :elapsed elapsed-ms)
(log/warn :prompt-photo :retval retval)
retval))
@slipset
Copy link

slipset commented Mar 4, 2024

It is now trivial to add back the timings under the various passes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment