Skip to content

Instantly share code, notes, and snippets.

@mkmik
Created March 1, 2010 16:04
Show Gist options
  • Save mkmik/318486 to your computer and use it in GitHub Desktop.
Save mkmik/318486 to your computer and use it in GitHub Desktop.
(ns authfile.lazy-couch
(:use couchdb.client))
(defn lazy-couch
"transforms a function which takes a couch-db parameter map into a lazy streaming function:
(defn lazy-get-all-record-ids [] (lazy-couch get-all-record-ids))
(defn lazy-get-all-jsons-raw [] (lazy-couch get-all-jsons-raw :key))
It works only on single argument functions. I normally define my own helpers on top low leve clojure-couchdb accessors.
It works only for views which have unique keys, otherwise you could get some duplicates."
([f]
(lazy-couch f identity))
([f get-key]
(lazy-couch f get-key 100))
([f get-key chunk-size]
(let [lazy-fetcher (fn lazy-fetcher
([lastkey]
(let [head (f {:limit chunk-size :startkey (get-key lastkey) :skip 1})]
(if (> chunk-size (count head))
head
(lazy-cat head (lazy-fetcher (last head))))))
([]
(let [head (f {:limit chunk-size})]
(lazy-cat head (lazy-fetcher (last head))))))]
(lazy-fetcher))))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment