Skip to content

Instantly share code, notes, and snippets.

@ericnormand
Last active September 16, 2019 02:37
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ericnormand/0cf9051444599c746d2f92e0227e578d to your computer and use it in GitHub Desktop.
Save ericnormand/0cf9051444599c746d2f92e0227e578d to your computer and use it in GitHub Desktop.

make an action idempotent

Clojure.core has a function called memoize. You probably already know what it does, but just to be on the same page, memoize takes a function and returns a new function. The new function is just like the argument you pass it, but it has a superpower. It's now cached. It will remember any arguments you pass it, and the return value it found, and give it to you without doing all the work. Of course, it has to do the work the first time, but after that, it's all cached.

You can do this same thing, but instead of giving a function a superpower of being cached, you can make it idempotent. Let's say I had a function that connected to an SMTP server to send an email. I don't want to send the email twice, but I want to be able to retry in case it didn't succeed. So I make the action idempotent so the duplicates don't matter. And what does it mean to be the "same email"? There needs to be a notion of identity.

Write a function called idempotent-by that takes a function and returns a new function that is like the original except calling it twice on the same thing does the action only once. The first argument to idempotent-by, before the function in question, should be another function that defines the key. Just like group-by takes a keyfn, idempotent-by takes a keyfn.

(defn idempotent-by [keyfn f]

  )
;; Handle exceptions by swapping only after successfully returning from f
;; Use a lock to make it transactional
;; Based on Mark Champine's code
(defn idempotent-by
[keyfn f]
(let [mymemory (volatile! #{})]
(fn [& arglist]
(let [k (apply keyfn arglist)]
(locking mymemory
(when (not (contains? @mymemory k))
(let [ans (apply f arglist)]
(vswap! mymemory conj k)
ans)))))))
(defn idempotent-by
"modify f so that calls to it are idempotent for identical inputs.
keyfn identifies identical inputs"
[keyfn f]
(let [mymemory (atom #{})] ;; per function memory atom
(fn [& arglist]
(let [k (apply keyfn arglist)
[ov _] (swap-vals! mymemory conj k)]
(when (not (contains? ov k)) ;; if we have never seen (keyfn args)
(apply f arglist)))))) ;; then call as normal, else do nothing.
;; Test 1: identity depends only on # of args passed
(defn doit [& al] (str "never saw " (count al) " args passed before."))
(defn count-keyfn [& ks] (count ks))
(def idpdoit-count (idempotent-by count-keyfn doit))
;; test run
(idpdoit-count :a :b)
;; "never saw 2 args passed before."
(idpdoit-count :a :b)
;; nil
(idpdoit-count :a :b :c)
;; "never saw 3 args passed before."
;; Test 2: identity on list of strings is case-insensitive
(defn doit2 [& al] (str "got this arg list: " al))
(defn lc-keyfn [& ks] (map clojure.string/lower-case ks))
(def idpdoit-lc (idempotent-by lc-keyfn doit2))
;; test run
(idpdoit-lc "Foo")
;; "got this arg list: (\"Foo\")"
(idpdoit-lc "foo")
;; nil
(idpdoit-lc "foo2")
;; "got this arg list: (\"foo2\")"
(idpdoit-lc "foo" "Bar")
;; "got this arg list: (\"foo\" \"Bar\")"
(idpdoit-lc "Foo" "bar")
;; nil
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment