Skip to content

Instantly share code, notes, and snippets.

@maxrothman
Last active February 24, 2022 15:25
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save maxrothman/f91ffccd8f2ba22618b2c699c8a8fbb1 to your computer and use it in GitHub Desktop.
Save maxrothman/f91ffccd8f2ba22618b2c699c8a8fbb1 to your computer and use it in GitHub Desktop.

A problem with errors in web API handlers

If you're building an web API server, you probably have handlers near the top of your application that look something like this. Some of the code may be in middleware or interceptors rather than in the handlers themselves, but that's more of a change of address than change of approach.

(defn handler [request]
  (try
    (let [r1 (query-something (get-in request [:query-params :a]))
          r2 (query-something-else (:body request))
          _ (do-something r1 r2)
          r3 (do-something-else r2)]
      {:status 200
       :body {:a (:a r1)
              :b (:b r2)
              :c (:c r3)}})
    (catch ExceptionInfo e
      (case (:type (ex-data e)
        ::db/foo-doesnt-exist {:status 404 :body "I'd rather you didn't"}
        ::domain/something-specific {:status 400 :body "You can't frob a bar"}
        ::other-domain/also-specific {:status 400 :body (str "Nor can you baz a "
                                                             (-> e ex-data :val))})))
    (catch SomethingElseNotAnExceptionInfo e
      {:status 401 :body "Please stop"})))

Now, that looks pretty messy, right? Lots of keyboard ink has been spilled trying to improve the situation, but let's try applying the popular ex library:

(defn handler [request]
  (ex/try+
    (let [r1 (query-something (get-in request [:query-params :a]))
          r2 (query-something-else (:body request))
          _ (do-something r1 r2)
          r3 (do-something-else r2)]
      {:status 200
       :body {:a (:a r1)
              :b (:b r2)
              :c (:c r3)}})
    (catch ::db/foo-doesnt-exist e
      {:status 404 :body "I'd rather you didn't"})
    (catch ::domain/something-specific e
      {:status 400 :body "You can't frob a bar"})
    (catch ::other-domain/also-specific e
      {:status 400 :body (str "Nor can you baz a " (-> e ex-data :val))})
    (catch SomethingElseNotAnExceptionInfo e
      {:status 401 :body "Please stop"})))

While ex does improve the syntax a little, it doesn't resolve the big problems with this code. Let's take a step back and think about the purpose of error handling in a web API server.

What happens outside of the happy path is part of the behavior specification of an application. If something went wrong, then the application obviously shouldn't return a 200 OK, but returning a 500 for all exceptional behavior isn't desirable either. More generally, each non-happy path in an application should be mapped to a specific response.

It's possible to apply blanket rules to certain classes of non-happy-paths. If the user isn't authenticated we should return a 401, and if they send us malformed JSON we should return a 400. Those are a great fit for middleware/interceptors, and of course it makes sense to have a catchall exception handler at the top of your application that maps anything unhandled to a 500. But other non-happy paths are more domain-specific and only make sense in the context of certain endpoints.

That brings us back to the code above. The main problem with it is that at some point, it will probably change. When it does, will the dev who changes it still know whether all the necessary exceptions are still being caught, or that all of the exceptions being caught can still be thrown by that code?

Perhaps those questions would be easier to answer if it was easier to tell which parts of the function threw which exceptions:

(defn handler [request]
  (e/trys'
    (let [r1 (e/try' (query-something (get-in request [:query-params :a]))
                     {::db/foo-doesnt-exist (constantly
                                             {:status 404
                                              :body "I'd rather you didn't"})})
          r2 (query-something-else (:body request))
          _ (e/try' (do-something r1 r2)
                    {::domain/something-specific (constantly 
                                                  {:status 400
                                                   :body "You can't frob a bar"})})
          r3 (e/try' (do-something-else r2)
                     {::other/domain-specific {:status 400
                                               :body #(str "Nor can you baz a "
                                                           (-> % ex-data :val))}
                      SomethingElseNotAnExceptionInfo (constantly
                                                       {:status 401
                                                        :body "Please stop"})})]
      {:status 200
       :body {:a (:a r1)
              :b (:b r2)
              :c (:c r3)}})))

Setting aside the awful names of try' and trys', looking at this code, I instantly know which parts of the function can throw which exceptions and map onto which responses. By decreasing the distance between cause and effect, this code gets easier to reason about and maintain. Another way of thinking about it is that try/catch does two things: it maps exceptions to values, and it allow for early return. This approach decomplects those: try' maps exceptions to values, and trys' marks where an early return can happen from.

A note about monads

Some people, when faced with the same problems, have applied a monadic or "railway-oriented" approach, which looks something like this:

;; Try-catch like
(f/attempt-all [a (do-a)
                b (do-b)
                _ (do-c)]
  {:status 200
   :body {:a a :b b}}
  (f/when-failed [e]
    (cond (:type e)
      ...)))

;; Or thread-like
(either-> x
          do-a   [handle-a-fail]
          do-b   [handle-b-fail]
          do-c)) [handle-c-fail]

Of those two syntaxes, the former does not differ significantly from normal exceptions, and the latter cannot express all the control flows you might need in your application. I think the insight here is that returning errors rather than throwing them doesn't fundamentally change the nature of error handling: in a non-strictly-typed language you still need a human to make sure your top-level code is in sync with what errors could be raised below it in the call stack.

Besides, with the building blocks presented above, writing an either-like threading macro is relatively straightforward.

(defn whoops [x]
  (throw (ex-info "hi" {:type :foo})))

(try-> 1
       whoops {:foo (constantly "got it")}
       (+ 1))
;; => "got it"

Notes

  • The code below only supports catching ExceptionInfos. Supporting general exceptions should be relatively straightforward, using ex under the hood would make that easy to do.
  • Use this code at your own risk! I haven't tested it in production, and I'm just some guy on the internet.
(ns exceptions.core
(:import exceptions.CustomException))
(defmacro trys'
"Marks a context for early return. If any exceptions are caught by a try'
inside a trys', the trys' will return the specified result."
[& forms]
`(try
~@forms
(catch CustomException e#
(:val (.getData e#)))))
(defmacro try'
"Marks a possible point of early return. If an ExceptionInfo is thrown within
'form' with a :type that's a key of 'ex-map', the corresponding value of
'ex-map' is returned from the trys' wrapping this form."
[form ex-map]
`(try
~form
(catch clojure.lang.ExceptionInfo e#
(if-let [resp# (get ~ex-map (:type (ex-data e#)))]
(throw (CustomException. {:val (resp# e#)}))
(throw e#)))))
(defmacro try->
"Thread-first x through the first of each pair of 'forms'. If any of these
throw an ExceptionInfo, handle it as if wrapped in try' with the second of
each pair as the ex-map. Returns either the first mapped exception value
or x threaded through all the forms if no exceptions are thrown."
[x & forms]
`(trys'
~(loop [x x, [[form handlers] & rest] (partition 2 forms)]
(if form
(let [threaded (if (seq? form)
`(try'
~(with-meta
`(~(first form) ~x ~@(next form))
(meta form))
~handlers)
`(try' (~form ~x)
~handlers))]
(recur threaded rest))
x))))
(ns exceptions.CustomException
"Custom exception type to make sure trys' doesn't accidentally catch unhandled exceptions"
(:gen-class
:extends java.lang.Error
:constructors {[clojure.lang.IPersistentMap] []} ; mapping of my-constructor -> superclass constuctor
:init init
:methods [[getData [] clojure.lang.IPersistentMap]]
:state state ; name for the var that holds your internal state
:main false))
(defn -init [context]
;; first element of vector contains parameters for the superclass constructor
;; Second element will be your internal state
[[] context])
(defn -getData [this]
(.state this)) ; accessing the internal state
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment