Skip to content

Instantly share code, notes, and snippets.

@gerritjvv
gerritjvv / gist:2dc9322f5cd1d29a4505
Created December 9, 2014 12:19
ApplyClojureRecords
(defrecord MyItem [a b c])
(apply ->MyItem ["1" "2" "3"])
;; #user.MyItem{:a "1", :b "2", :c "3"}
(map #(apply ->MyItem %) [ [1 2 3] [4 5 6] [7 8 9]])
;;(#user.MyItem{:a 1, :b 2, :c 3} #user.MyItem{:a 4, :b 5, :c 6} #user.MyItem{:a 7, :b 8, :c 9})
@gerritjvv
gerritjvv / gist:3b0e938bdcf91d2a450d
Created November 23, 2014 17:01
NPETypeHintFix
(defn safe-inc [l ^long x]
(if l
(+ ^long l x)
x))
// Compiled from form-init2140241509718408406.clj (version 1.5 : 49.0, super bit)
public final class no.disassemble$eval1752$safe_inc__1753 extends clojure.lang.AFunction implements clojure.lang.IFn$OLO {
// Field descriptor #9 Lclojure/lang/Var;
public static final clojure.lang.Var const__0;
@gerritjvv
gerritjvv / bootstrap.sh
Created September 2, 2014 08:57
Cassandra Vagrant bootstrap
!/usr/bin/env bash
NODE="$1"
echo "Bootstrapping node $1"
function install_setup(){
yum update
yum -y install wget vim snappy-devel snappy lzo lzop elinks glibc.i686
}
@gerritjvv
gerritjvv / Vagrant
Created September 2, 2014 08:54
Cassandra Vagrant File
# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.provider "virtualbox" do |v|
v.customize ["modifyvm", :id, "--memory", "2048"]
@gerritjvv
gerritjvv / merge-stats-map.clj
Last active August 29, 2015 14:01
The map is formed of key = topic, value = stats map of numeric stats The task was to aggregate the numeric values in the stats map for similar topics, each message could have multiple topics.
(def msgs [ {"a" {:stat1 1 :stat2 10} "b" {:stat1 1}} {"a" {:stat1 1}}])
(def merged (apply merge-with (partial merge-with +) msgs))
;; {"a" {:stat1 2, :stat2 10}, "b" {:stat1 1}}
(prn merged)
;; {"a" {:stat1 2, :stat2 10}, "b" {:stat1 1}}
(= (-> merged first second :stat1) 2)
;; true
(require '[clojure.core.async :refer [go-loop chan <! close!]])
(require '[fun-utils.core :refer [go-seq]])
;When writing go blocks there is a repeating pattern to avoid 100% cpu spin when the channel is closed.
;e.g.
;if we run the below code, the loop will spin without pause.
;;running this code will cause nil to be printed without pause
@gerritjvv
gerritjvv / clojure async channel bridge
Created December 31, 2013 16:58
Simple thing to do, but requires repeated typing. The only thing it does is move from channel-a to channel-b. This function is implemented in https://github.com/gerritjvv/fun-utils
(require '[clojure.core.async :refer [chan go >! <! >!! <!!]])
(require '[clojure.core.async :as async])
(defn chan-bridge
([ch-source map-f ch-target]
"map map-f onto ch-source and copy the result to ch-target"
(chan-bridge (async/map map-f [ch-source]) ch-target))
([ch-source ch-target]
"in a loop read from ch-source and write to ch-target
this function returns inmediately and returns the ch-target"
@gerritjvv
gerritjvv / buff-chan
Last active January 1, 2016 15:29
There are many times where you need to buffer up a series off results and then perform an operation on them, and if the count is not reached on a predefined timeout do the operation with the results collected. For my use case I'm writing a kafka producer and want messages received on the a send function to be buffered before sending to kafka. Th…
(require '[clojure.core.async :refer [go alts! >! <! >!! <!! chan timeout]])
(defn buffered-chan
"Reads from ch-source and if either timeout or the buffer-count has been
read the result it sent to the channel thats returned from this function"
([ch-source buffer-count timeout-ms]
(buffered-chan ch-source buffer-count timeout-ms 1))
([ch-source buffer-count timeout-ms buffer-or-n]
(let [ch-target (chan buffer-or-n)]
(go
@gerritjvv
gerritjvv / queue offer
Created November 2, 2013 22:54
A offer function that will timeout if the offer cannot be made
(use 'clojure.core.async)
(defn offer [ch msg timeout-ms]
"Blocks till the message can be placed on the queue and returns true otherwise returns false"
(not
(nil?
(first (alts!! [(go (>! ch msg) msg) (timeout timeout-ms)])))))
;;lets test
@gerritjvv
gerritjvv / core.async-lazy-sequence
Created October 30, 2013 11:07
This is for a situation where you have N amount of threads reading from different sources and want to consume all of the data they produce as a single sequence. Can be described as merging N queues from different sources and works well when the data produced is from IO. e.g. My usage is with kafka, I have multiple kafka topics and partitions to …
(use 'clojure.core.async)
;this is the function you want to use
(defn lazy-channels [chs]
(lazy-seq (cons (let [ [v _] (alts!! chs)] v) (lazy-channels chs))))
;now for the tesging
(def chs [ (chan) (chan) (chan) ]) ; the channels can come from anywhere, here we are using three channels for testing