Skip to content

Instantly share code, notes, and snippets.

@pedro pedro/enqueue.r
Last active Aug 29, 2015

Embed
What would you like to do?
Running redis-based workers in R on Heroku
# same code used in https://gist.github.com/pedro/f931e8cd20b5770b6780
# only using doRedis instead of doParallel
library(foreach)
library(doRedis)
redis_url <- Sys.getenv("REDIS_URL")
host <- gsub("redis://.*@(.+):.*", "\\1", redis_url)
pass <- gsub("redis://.*:(.+)@.*", "\\1", redis_url)
port <- as.integer(gsub(".*:(.*)$", "\\1", redis_url))
registerDoRedis("jobs", host=host, port=port, password=pass)
x <- iris[which(iris[,5] != "setosa"), c(1,5)]
trials <- 10000
ptime <- system.time({
r <- foreach(icount(trials), .combine=cbind) %dopar% {
ind <- sample(100, 100, replace=TRUE)
result1 <- glm(x[ind,2]~x[ind,1], family=binomial(logit))
coefficients(result1)
}
})[3]
print(ptime)
# used by the R buildpack to install dependencies:
# https://github.com/virtualstaticvoid/heroku-buildpack-r
install.packages("foreach", dependencies = TRUE)
install.packages("rredis", dependencies = TRUE)
install.packages("doRedis", dependencies = TRUE)
$ heroku run Rscript enqueue.r
# in another window:
$ heroku logs --tail
Processing job 3 from queue jobs
Processing task 225 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 1325 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 7810 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 2226 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 5289 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 1963 ... from queue jobs jobID 3
Processing job 3 from queue jobs
Processing task 7053 ... from queue jobs jobID 3
...
library(doRedis)
redis_url <- Sys.getenv("REDIS_URL")
host <- gsub("redis://.*@(.+):.*", "\\1", redis_url)
pass <- gsub("redis://.*:(.+)@.*", "\\1", redis_url)
port <- as.integer(gsub(".*:(.*)$", "\\1", redis_url))
redisWorker("jobs", host=host, port=port, password=pass)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.