This is not a introduction post to Clojure transducer. Instead there are a lots of great introduction post out there. This post aims to clarify the greatness of transduer. In the introduction blog, this is how they describe:
Transducers are composable algorithmic transformations. They are independent from the context of their input and output sources and specify only the essence of the transformation in terms of an individual element. Because transducers are decoupled from input or output sources, they can be used in many different processes - collections, streams, channels, observables, etc. Transducers compose directly, without awareness of input or creation of intermediate aggregates.
This post will help you understand the above statement.
For me transducer does 2 great things
- Parallel process
- Independence input source, therefore reuseable
Think about typical mapping. Let's say we have a list of number which represent our company payout.
(def January-payout
[10, 5, 15, 20, 4])Before payout, we also need to include a few taxes.
(defn payout-with-tax
[xs]
(->>
xs
(map add-goverment-tax)
(map add-service-tax)))Normally, you won't do this because it loop 2 times. Instead we will either
(def add-all-taxes (comp add-goverment-tax))
(defn payout-with-tax
[xs]
(->>
xs
(map add-all-taxes)))Or,
(defn payout-with-tax
[xs]
(->>
xs
(map #(-> % add-goverment-tax add-service-tax))))We make sure we don't do extra loop. Then what if your company just decide to not transfer the payouts which is under 10. Easy,
(defn great-payout-with-tax
[xs]
(->>
xs
(filter #(> % 10))
(map add-all-taxes)))Now we have 2 loops. What if I tell you we can make it into 1 loop with transducer ?
In Clojure if you pass in one argument to map or filter, they return a transduer.
(You can check the source of map with (source map))
(defn add-all-taxes
[x]
(* x 1.08))
(defn great-payout?
[x]
(> x 10))
;; our transducer
(def payoutForm
(comp
(filter great-payout?) ;; <-- a transducer
(map add-all-taxes))) ;; <-- also a transducer
(defn great-payout-with-tax
[xs]
(->>
xs
(into [] payoutForm)))
;; or
(defn great-payout-with-tax
[xs]
(into [] payoutForm xs))By now, we have 1 loop back =)
We used into [] to build our payoutForm transformation into a collection but as you can see, our payoutForm transducer doesn't really care about it. That's what I mean independence input source.
Basically, our payoutForm doesn't care what is the input source and how we process the final output. In the example above, our input came from January-payout which is a normal list while our output is another list built up from into [].
We can also process lazy-seq.
(def lazy-January-payout
(lazy-seq January-payout))
(take 1 (sequence payoutForm lazy-January-payout))The possibility is infinite. Channel can also be the input:
(let [chan (async/chan 1 payoutForm)]
(async/take! chan println)
(async/put! chan 15))
;; 16.200000000000003Our payoutForm can be reused to do other thing, like summarizing the total payout of month.
(reduce (payoutForm +) 0 January-payout)
;; 37.800000000000004
;; don't forget Clojure has a transduce function
(transduce payoutForm + 0 January-payout)
;; 37.800000000000004Or we can build up another process to sum up many payout:
(def January-payout
[10, 5, 15, 20, 4])
(def February-payout
[100, 50, 5, 7, 40])
(def payoutForm
(comp
(filter great-payout?)
(map add-all-taxes)))
(def payoutsForm
(comp
(mapcat identity)
payoutForm))
(transduce payoutsForm + 0 [January-payout February-payout])
;; 243.0Tada ! Building up example like this is totally easy but in real life is transducer really will be useful ? Indeed. Hopefully I can explore a more real world like example in my side project.
Reference: