Skip to content

Instantly share code, notes, and snippets.

@ghadishayban
Last active November 20, 2021 00:06
Show Gist options
  • Save ghadishayban/f905be564d1d37ba9fa4d77b0d5e8848 to your computer and use it in GitHub Desktop.
Save ghadishayban/f905be564d1d37ba9fa4d77b0d5e8848 to your computer and use it in GitHub Desktop.
unified generators
;;
;; Example usages at the bottom of the file
;;
(defn productions
"Returns a sequence of values by repeatedly calling `produce!` until it
returns `fin`. The sequence can be used lazily/caching or reducible/non-caching.
The arity-2 variant's `produce!` takes no arguments and returns a value
or the terminator.
The arity-3 variant takes an initial state `s`, and its `produce!` function
takes the state and returns the next state or terminator. Each state will appear
in the sequence."
([produce! fin]
(reify
clojure.lang.Sequential
clojure.lang.Seqable
(seq [_]
((fn step-fn []
(let [val (produce!)]
(if (identical? fin val)
nil
(cons val (lazy-seq (step-fn))))))))
IReduceInit
(reduce [_ rf init]
(loop [res init]
(let [v (produce!)]
(if (identical? fin v)
res
(let [res (rf res v)]
(if (reduced? res)
@res
(recur res)))))))))
([s produce! fin]
(reify
clojure.lang.Sequential
clojure.lang.Seqable
(seq [_]
((fn step-fn [s]
(if (identical? fin s)
nil
(cons s (lazy-seq (step-fn (produce! s))))))
s))
clojure.lang.IReduceInit
(reduce [_ rf init]
(loop [acc init
s s]
(if (identical? fin s)
acc
(let [acc (rf acc s)]
(if (reduced? acc)
@acc
(recur acc (produce! s))))))))))
;;;; two arity
;; (defn faster-line-seq
;; [^BufferedReader rdr]
;; (productions #(or (.readLine rdr) ::eos) ::eos))
;;;; three-arity w/ internal state
;; user> (vec (productions 10 (fn [i]
;; (if (> i 5) (dec i) :fin)) :fin))
;; [10 9 8 7 6 5]
;; this arity is useful for expressing things like paginating API calls
@ghadishayban
Copy link
Author

In the iterate* + take-while the terminator ends up being produced then filtered by take-while. In this proposal the terminator never appears in the sequence, it just avoids production.

I think the arity-2 is uncontroversial, but wrt to the state carrying arity: expressing the unfolding in layers using iterate + transducer obscures nature of the job.

; arity 3
(productions
  (api)
  (fn [result]
    (if (good? result)
      (api (:next result))
    ::eof)
  ::eof)

; arity 2
(productions #(or (read-fressian rdr) ::eos) ::eos)

@arichiardi
Copy link

Small typo on line 25, it should be clojure.lang.IReduceInit

@arichiardi
Copy link

arichiardi commented Dec 21, 2016

@ghadishayban I tried and it looks like if I do (take 2 (productions ....)) it continues over and over without actually realizing only just two batches.
I will try to investigate the problem.

EDIT: as customary, it was on my side

@ghadishayban
Copy link
Author

ghadishayban commented Apr 16, 2017

new version

(defn supply
  "Produces a sequence of values by repeatedly calling `f` for side-effects
    until it returns `fin`. The sequence can be used lazily/caching or
   reducible/non-caching. Not guaranteed to produce any values"
  [f fin]
  (reify
    clojure.lang.Seqable
    (seq [_]
      ((fn step []
         (let [v (f)]
           (if (identical? v fin)
             nil
             (cons v (new clojure.lang.LazySeq step)))))))
    clojure.lang.IReduceInit
    (reduce [_ rf init]
      (loop [res init]
        (let [v (f)]
          (if (identical? v fin)
            res
            (let [res (rf res v)]
              (if (reduced? res)
                @res
                (recur res)))))))
    clojure.lang.Sequential))

;; differences from scheme unfold
;; even initial value is lazy
;; predicate sense reversed
;; internal state == produced value, no special mapper-fn
;; no tail-gen
(defn series
  "Produces a sequence of values.

   `f` is a function that given a value, returns the next value.
   `continue?` is a predicate that determines whether to produce
    the next value. `f` called with no arguments produces the
    initial value. Always produces at least one value."
  [f continue?]
  (reify
    clojure.lang.Seqable
    (seq [_]
      ((fn step [seed]
         (cons seed
               (when (continue? seed)
                 (lazy-seq (step (f seed))))))
       (f)))
    clojure.lang.IReduceInit
    (reduce [_ rf init]
       (loop [seed (f)
             ret (rf init seed)]
        (if (reduced? ret)
          @ret
          (if (continue? seed)
            (let [next (f seed)]
               (recur next (rf ret next)))
            ret))))
    clojure.lang.Sequential))

Usage

;; supply
(supply #(or (.readLine ^BufferedReader rdr) ::eof) ::eof)

;; series
(let [api (fn ([]
	        (GET "/initial"))
	      ([response]
	        (GET (next-link response))))]
	(series api next-link))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment