Skip to content

Instantly share code, notes, and snippets.

@johnmarinelli
Created January 24, 2016 04:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save johnmarinelli/a8f3be10a7f994bfb532 to your computer and use it in GitHub Desktop.
Save johnmarinelli/a8f3be10a7f994bfb532 to your computer and use it in GitHub Desktop.
(defn insert-page-into-db [page]
(println page) ; <-- doesn't get run
(try
(sql/insert! db-conn-vars :pages
[:title, :pageid]
[(get page "title"), (Integer. (get page "pageid"))]) (catch Exception e (println e))))
(defn insert-pages-into-db [pages]
(println pages) ; <-- gets ran
(map
(fn [page]
(insert-page-into-db page)) pages))
(defn get-wikimedia-pages [from]
(let [body (get-body (get-json-response (build-url from)))]
(let [continue-from (url-encode (get-continue-from body)) pages (get-pages body)]
(insert-pages-into-db pages)
(dosync (println from))
(if (not (= "%22Do_not_disturb%22_sign" continue-from))
(recur continue-from)))))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment