Skip to content

Instantly share code, notes, and snippets.

@danielpcox
Created March 11, 2015 21:21
Show Gist options
  • Save danielpcox/c70a8aa2c36766200a95 to your computer and use it in GitHub Desktop.
Save danielpcox/c70a8aa2c36766200a95 to your computer and use it in GitHub Desktop.
Simple, recursive deep-merge in Clojure.
(ns deep-merge-spec
(:require [midje.sweet :refer :all]
[util :as u]))
(fact (u/deep-merge {:one 1 :two 2}
{:one 1 :two {}})
=> {:one 1 :two {}})
(fact (u/deep-merge {:one 1 :two {:three 3 :four {:five 5}}}
{:two {:three {:test true}}})
=> {:one 1 :two {:three {:test true} :four {:five 5}}})
(fact (u/deep-merge {:one {:two {:three 3}}
:four {:five {:six 6}}}
{:one {:seven 7 :two {:three "three" :nine 9}}
:four {:eight 8 :five 5}
:ten 10})
=> {:one {:two {:three "three" :nine 9}
:seven 7}
:four {:five 5 :eight 8}
:ten 10})
(fact (u/deep-merge {:one {:two 2 :three 3}}
{:one {:four 4 :five 5}})
=> {:one {:two 2 :three 3 :four 4 :five 5}})
(fact (u/deep-merge {:one 1 :two {:three 3}}
{:one 2 :two {:three 4}}
{:one 3 :two {:three 5}}
{:one 4 :two {:three 6}})
=> {:one 4 :two {:three 6}})
(ns util)
(defn deep-merge [v & vs]
(letfn [(rec-merge [v1 v2]
(if (and (map? v1) (map? v2))
(merge-with deep-merge v1 v2)
v2))]
(when (some identity vs)
(reduce #(rec-merge %1 %2) v vs))))
@dijonkitchen
Copy link

@cjsauer actually your script didn't work at all. You have to return (last vs) not v:

(defn deep-merge [v & vs]
  (letfn [(rec-merge [v1 v2]
            (if (and (map? v1) (map? v2))
              (merge-with deep-merge v1 v2)
              v2))]
    (if (some identity vs)
      (reduce #(rec-merge %1 %2) v vs)
      (last vs))))

e.g:

(deep-merge {:a {:b true}} {:a {:b false}} {:a {:b nil}})
; with your script: 
{:a {:b true}}
; with mine: 
{:a {:b nil}}

@Freezystem you're right that your deep-merge works with nil values, but not nil maps.

Example:

(deep-merge {:a 1} nil)
nil ;; but it should be {:a 1}

@cjsauer's works for nil maps, but not nil values as you noted.

Is there a way to reconcile the two and be able to handle both nil maps and maps with nil values?

@fl00r
Copy link

fl00r commented Sep 20, 2018

(defn- deep-merge [& maps]
  (let [merge-fn (fn *merge* [& args]
                   (if (every? map? args)
                     (apply merge-with *merge* args)
                     (last args)))]
    (apply merge-with merge-fn maps)))
(deep-merge {:a 1} nil)
#=> {:a 1}
(deep-merge {:a {:b true}} {:a {:b false}} {:a {:b nil}})
#=> {:a {:b nil}}

or slightly more concise

(defn deep-merge [& maps]
  (apply merge-with (fn [& args]
                      (if (every? map? args)
                        (apply deep-merge args)
                        (last args)))
         maps))

@loganpowell
Copy link

loganpowell commented Oct 10, 2018

@fl00r this was genius. It's not only more concise, but it's about 7x faster. I brought down an @ 100mb deep-merge from @ 15 minutes to @ 2 minutes.

Thank you all so much. Progress is made!

@loganpowell
Copy link

loganpowell commented Nov 14, 2018

Courtesy of @cgrand

(defn deep-merge
  [a b]
  (if (map? a)
    (into a (for [[k v] b] [k (deep-merge (a k) v)]))
    b))

@loganpowell
Copy link

loganpowell commented Nov 14, 2018

And @jaihindhreddy

(defn deep-merge [a b]
  (if (map? a)
    (merge-with deep-merge a b)
    b))

and variadic:

(defn deep-merge [a & maps]
  (if (map? a)
    (apply merge-with deep-merge a maps)
    (apply merge-with deep-merge maps)))

@thobbs
Copy link

thobbs commented Feb 22, 2019

Note that the second, variadic version of deep-merge provided here has unusual/incorrect behavior for false-y values. For example:

dev=> (deep-merge {:a nil} {:a false})
{:a nil}

This is due to merge-with checking for at least one "truthy" argument:

(defn merge-with
  "Returns a map ..."
  [f & maps]
  (when (some identity maps)
    (let [merge-entry (fn [m e]
      ...
      (reduce1 merge2 maps))))

Which produces behavior like:

dev=> (merge-with first false nil)
nil

@dijonkitchen
Copy link

After using deep-merge for a bit, I think I know why it doesn't already exist. It encourages difficult to understand nested maps when flat maps are simpler/better. Perhaps its absence speaks to it as a code smell.

@andrewboltachev
Copy link

@fl00r Here's the problem (if that makes sense still):

cljs.user=> (defn deep-merge [& maps]
       #_=>   (apply merge-with (fn [& args]
       #_=>                       (if (every? map? args)
       #_=>                         (apply deep-merge args)
       #_=>                         (last args)))
       #_=>          maps))
       #_=> 
#'cljs.user/deep-merge
cljs.user=> (deep-merge {:m {:a 1 :b 2}} {:m nil} {:m {:a 2 :c 3}})
{:m {:a 2, :c 3}}
cljs.user=> (deep-merge {:m {:a 1 :b 2}} {:m  {}} {:m {:a 2 :c 3}})
{:m {:a 2, :b 2, :c 3}}

So I guess sth like this:

(defn deep-merge [& maps]
  (apply merge-with (fn [& args]
                      (if (every? #(or (map? %) (nil? %)) args)
                        (apply deep-merge args)
                        (last args)))
         maps))

@fl00r
Copy link

fl00r commented Jan 10, 2020

@andrewboltachev I would argue that this behavior is an expected one.
nil is a legit value, so (deep-merge {:m {:a 1 :b 2}} {:m nil}) => {:m nil}

cheers

@andrewboltachev
Copy link

@fl00r well, my argument here is that, when using just merge:

user=> (merge {:a 1 :b 2} nil)
{:a 1, :b 2}

and I think of nil as kinda "absence of value"

@fl00r
Copy link

fl00r commented Jan 10, 2020

@andrewboltachev

but

(merge {:a 1} {:a nil})
{:a nil}

In deep merge I would prefer this semantic.

But that is of course a matter of taste.

And as @dijonkitchen mentioned

After using deep-merge for a bit, I think I know why it doesn't already exist. It encourages difficult to understand nested maps when flat maps are simpler/better. Perhaps its absence speaks to it as a code smell.

Of course you can modify the code to fit your needs.

@JasonStiefel
Copy link

JasonStiefel commented Sep 25, 2020

@juman123
Copy link

juman123 commented Jan 4, 2024

Here's one more version.

This one is a lot less concise, but potentially easier to understand. It might also perform better than some other versions, since I'm using loops and recurs.

(defn merge-entries [value-fn into-map entries]
  (loop [result    into-map
         remaining (seq entries)]
    (if-not remaining
      result
      (let [entry   (first remaining)
            k       (key entry)
            new-val (val entry)
            cur-val (get result k)]
        (recur (if (nil? cur-val)
                 (assoc result k new-val)
                 (assoc result k (value-fn cur-val new-val)))
               (next remaining))))))


(defn merge-maps [value-fn & maps]
  (loop [results   nil
         remaining (seq maps)]
    (if-not remaining
      results
      (recur (merge-entries value-fn (or results {}) (-> remaining first seq))
             (next remaining)))))


(defn deep-merge [& maps]
  (let [value-fn (fn [v1 v2]
                   (if-not (and (map? v1) (map? v2))
                     v2
                     (deep-merge v1 v2)))]
    (apply merge-maps value-fn maps)))


(deep-merge {:m {:a 1 :b 2}} {:m nil} {:m {:a 2 :c 3}})
(deep-merge {:m {:a 1 :b 2}} {:m {}} {:m {:a 2 :c 3}})
(deep-merge {:a {:b true}} {:a {:b false}} {:a {:b nil}})
(deep-merge {:a 1} nil)
(deep-merge {:a {:b true}} {:a {:b false}} {:a {:b nil}})
(deep-merge {:one 1 :two {:three 3}}
            {:one 2 :two {:three 4}}
            {:one 3 :two {:three 5}}
            {:one 4 :two {:three 6}})

{:m {:a 2, :c 3}}
{:m {:a 2, :b 2, :c 3}}
{:a {:b nil}}
{:a 1}
{:a {:b nil}}
{:one 4, :two {:three 6}}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment