UseR 2019 tutorial - Get up to speed with Bayes test script

get_up_to_speed_with_bayes_test_script.R

# Prior to the tutorial make sure that the script below runs without error on your R installation.
# You first need to install the follwoing packages:
# install.packages(c("rstanarm", "prophet", "CausalImpact"))

library(rstanarm)
library(prophet)
library(CausalImpact)

# This will test that rstanarm works
# Don't be alarmed if you get a warning about "divergent transitions "
fit <- stan_lm(mpg ~ wt + qsec + am, data = mtcars, prior = R2(0.75))
plot(fit, prob = 0.8)

# This tests that prophet is working
history <- data.frame(ds = seq(as.Date('2015-01-01'), as.Date('2016-01-01'), by = 'd'),
                      y = sin(1:366/200) + rnorm(366)/10)
m <- prophet(history)
future <- make_future_dataframe(m, periods = 365)
forecast <- predict(m, future)
plot(m, forecast)

# This tests that CausalImpact is working
# First simulating some data
x1 <- 100 + arima.sim(model = list(ar = 0.999), n = 52)
y <- 1.2 * x1 + rnorm(52)
y[41:52] <- y[41:52] + 10
data <- cbind(y, x1)
pre.period <- c(1, 40)
post.period <- c(41, 52)
# Then running CausalImpact
impact <- CausalImpact(data, pre.period, post.period)
plot(impact)

Having some trouble with the installation of CausalImpact. Boom and BoomSpikeSlab are installed and running but I simply can't make bsts to install, and CausalImpact depends on it. Any new resources? I've seen some discussion about the issue for Windows users, but nothing really for Linux users...

@keniatw

What error do you get and what platform are you on? Could you perhaps paste in the whole error message as a reply?

/Rasmus

Here is the error message I got:

• installing source package ‘prophet’ ...
  ** package ‘prophet’ successfully unpacked and MD5 sums checked
  ** libs
  xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
  ERROR: compilation failed for package ‘prophet’
• removing ‘/Library/Frameworks/R.framework/Versions/3.5/Resources/library/prophet’
  Warning in install.packages :
  installation of package ‘prophet’ had non-zero exit status

@yaoGithub2018

You are trying to install prophet "from source" which means you need to be able to compile it which requires you to install some stuff firsts (see here: https://support.rstudio.com/hc/en-us/articles/200486498-Package-Development-Prerequisites) .

However, when installing I believe you should be given the option to install an older version that is pre-compiled, that is completely fine, and I would go with that option if I were you. :)

Let me know how things works out for you!
/Rasmus

Thank you! All works out!

I am having trouble to install "prophet". I also tried to installed Xcode, but am not sure whether that was necessary and I was not able to
downloads and install the Command Line Tools...
This is part of the messages I got:

** package 'prophet' successfully unpacked and MD5 sums checked
Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) :
there is no package called 'StanHeaders'

• removing '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/prophet'
  Warning in install.packages :
  installation of package 'prophet' had non-zero exit status

library(prophet)
Error in library(prophet) : there is no package called 'prophet'

Do you have a solution?
Thanks, Erik

Hi @erikgiltay,
Can you install the StanHeaders package? Like this:

install.packages("StanHeaders")

Otherwise, do you get the option of installing an older version of prophet when you try to intall that does not require compilation from source? Then go for that option! :)

/Rasmus

Hi rasmus,

I was unable to install package prophet,

I also click on the link you shared to get support to install the package.

Below is the message I got

library(CausalImpact)
Error: package or namespace load failed for ‘CausalImpact’ in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]):
there is no package called ‘pillar’
library(prophet)
Error: package or namespace load failed for ‘prophet’ in loadNamespace(j <- i[[1L]], c(lib.loc, .libPaths()), versionCheck = vI[[j]]):
there is no package called ‘pillar’

Thank you! All works out!

Hi yaoGithub2018

Please how did you go about the instillation?

Kindly help out

Thanks

Please,

Hi all,

I have some problems installing rstanarm.

When I run install.packages("rstanarm") I get following errors:

ERROR: configuration failed for package ‘openssl’
* removing ‘/home/samuel/R/x86_64-pc-linux-gnu-library/3.4/openssl’
Warning in install.packages :
  installation of package ‘openssl’ had non-zero exit status
ERROR: dependency ‘openssl’ is not available for package ‘rsconnect’
* removing ‘/home/samuel/R/x86_64-pc-linux-gnu-library/3.4/rsconnect’
Warning in install.packages :
  installation of package ‘rsconnect’ had non-zero exit status
ERROR: dependency ‘rsconnect’ is not available for package ‘shinystan’
* removing ‘/home/samuel/R/x86_64-pc-linux-gnu-library/3.4/shinystan’
Warning in install.packages :
  installation of package ‘shinystan’ had non-zero exit status
ERROR: dependency ‘shinystan’ is not available for package ‘rstanarm’
* removing ‘/home/samuel/R/x86_64-pc-linux-gnu-library/3.4/rstanarm’
Warning in install.packages :
  installation of package ‘rstanarm’ had non-zero exit status

The downloaded source packages are in
	‘/tmp/RtmpGHcfvi/downloaded_packages’

I have tried running sudo apt-get install libcurl4-openssl-dev (suggested here https://discourse.mc-stan.org/t/install-issues-with-shinystan-with-r/1506 and sudo apt update && apt install libssl-dev (suggested here https://github.com/rocker-org/rocker/issues/124. I have also tried running install.packages(c("Rcpp", "RcppEigen", "rstan", "rstanarm")), accoring to the suggestions from https://github.com/stan-dev/rstanarm/issues/90. However, I still cant install rstanarm.

I am running R from RStudio, and I have the following system:

> version
               _                           
platform       x86_64-pc-linux-gnu         
arch           x86_64                      
os             linux-gnu                   
system         x86_64, linux-gnu           
status                                     
major          3                           
minor          4.4                         
year           2018                        
month          03                          
day            15                          
svn rev        74408                       
language       R                           
version.string R version 3.4.4 (2018-03-15)
nickname       Someone to Lean On         

Any ideas on what the problem can be?

(Installing the other packages worked fine for me).

Best,
Samuel

@SamuelWiqvist

It seems like it's not rstanarmthat is the problem, but the openssl dependency.
Does install.packages("openssl") work?
If not, that's where you would have to start digging... Sorry for not being more specific! :)

/Rasmus

@rasmusab Thanks for your comments!

I finally got rstanarm to work by before installing it running sudo apt install libssl-dev (see the comment related to Ubuntu here https://github.com/rocker-org/rocker/issues/124).

@rasmusab Is this the expected output? There is some output regarding "Disabling yearly seasonality." and warnings regarding "Removed 52 rows containing missing values (geom_path)."

> library(rstanarm)
> library(prophet)
> library(CausalImpact)
> 
> # This will test that rstanarm works
> # Don't be alarmed if you get a warning about "divergent transitions "
> fit <- stan_lm(mpg ~ wt + qsec + am, data = mtcars, prior = R2(0.75))

SAMPLING FOR MODEL 'lm' NOW (CHAIN 1).
Chain 1: 
Chain 1: Gradient evaluation took 0 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1: 
Chain 1: 
Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 1: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 1: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 1: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 1: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 1: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 1: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 1: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 1: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 1: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 1: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 1: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 1: 
Chain 1:  Elapsed Time: 3.998 seconds (Warm-up)
Chain 1:                2.404 seconds (Sampling)
Chain 1:                6.402 seconds (Total)
Chain 1: 

SAMPLING FOR MODEL 'lm' NOW (CHAIN 2).
Chain 2: 
Chain 2: Gradient evaluation took 0 seconds
Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 2: Adjust your expectations accordingly!
Chain 2: 
Chain 2: 
Chain 2: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 2: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 2: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 2: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 2: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 2: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 2: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 2: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 2: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 2: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 2: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 2: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 2: 
Chain 2:  Elapsed Time: 4.17 seconds (Warm-up)
Chain 2:                5.554 seconds (Sampling)
Chain 2:                9.724 seconds (Total)
Chain 2: 

SAMPLING FOR MODEL 'lm' NOW (CHAIN 3).
Chain 3: 
Chain 3: Gradient evaluation took 0 seconds
Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 3: Adjust your expectations accordingly!
Chain 3: 
Chain 3: 
Chain 3: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 3: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 3: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 3: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 3: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 3: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 3: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 3: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 3: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 3: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 3: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 3: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 3: 
Chain 3:  Elapsed Time: 3.83 seconds (Warm-up)
Chain 3:                2.556 seconds (Sampling)
Chain 3:                6.386 seconds (Total)
Chain 3: 

SAMPLING FOR MODEL 'lm' NOW (CHAIN 4).
Chain 4: 
Chain 4: Gradient evaluation took 0 seconds
Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds.
Chain 4: Adjust your expectations accordingly!
Chain 4: 
Chain 4: 
Chain 4: Iteration:    1 / 2000 [  0%]  (Warmup)
Chain 4: Iteration:  200 / 2000 [ 10%]  (Warmup)
Chain 4: Iteration:  400 / 2000 [ 20%]  (Warmup)
Chain 4: Iteration:  600 / 2000 [ 30%]  (Warmup)
Chain 4: Iteration:  800 / 2000 [ 40%]  (Warmup)
Chain 4: Iteration: 1000 / 2000 [ 50%]  (Warmup)
Chain 4: Iteration: 1001 / 2000 [ 50%]  (Sampling)
Chain 4: Iteration: 1200 / 2000 [ 60%]  (Sampling)
Chain 4: Iteration: 1400 / 2000 [ 70%]  (Sampling)
Chain 4: Iteration: 1600 / 2000 [ 80%]  (Sampling)
Chain 4: Iteration: 1800 / 2000 [ 90%]  (Sampling)
Chain 4: Iteration: 2000 / 2000 [100%]  (Sampling)
Chain 4: 
Chain 4:  Elapsed Time: 4 seconds (Warm-up)
Chain 4:                5.935 seconds (Sampling)
Chain 4:                9.935 seconds (Total)
Chain 4: 
Warning messages:
1: There were 13 divergent transitions after warmup. Increasing adapt_delta above 0.99 may help. See
http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup 
2: Examine the pairs() plot to diagnose sampling problems
 
> plot(fit, prob = 0.8)
> 
> # This tests that prophet is working
> history <- data.frame(ds = seq(as.Date('2015-01-01'), as.Date('2016-01-01'), by = 'd'),
+                       y = sin(1:366/200) + rnorm(366)/10)
> m <- prophet(history)
Disabling yearly seasonality. Run prophet with yearly.seasonality=TRUE to override this.
Disabling daily seasonality. Run prophet with daily.seasonality=TRUE to override this.
> future <- make_future_dataframe(m, periods = 365)
> forecast <- predict(m, future)
> plot(m, forecast)
> 
> # This tests that CausalImpact is working
> # First simulating some data
> x1 <- 100 + arima.sim(model = list(ar = 0.999), n = 52)
> y <- 1.2 * x1 + rnorm(52)
> y[41:52] <- y[41:52] + 10
> data <- cbind(y, x1)
> pre.period <- c(1, 40)
> post.period <- c(41, 52)
> # Then running CausalImpact
> impact <- CausalImpact(data, pre.period, post.period)
> plot(impact)
Warning messages:
1: Removed 52 rows containing missing values (geom_path). 
2: Removed 104 rows containing missing values (geom_path). 
> 

Yeah, that should be fine. As long as it runs! :) /R

On Thu, 4 Jul 2019 at 13:28, Johan Rex ***@***.***> wrote: @rasmusab <https://github.com/rasmusab> Is this the expected output? There is some output regarding "Disabling yearly seasonality." and warnings regarding "Removed 52 rows containing missing values (geom_path)." ` library(rstanarm) > library(prophet) > library(CausalImpact) > > # This will test that rstanarm works > # Don't be alarmed if you get a warning about "divergent transitions " > fit <- stan_lm(mpg ~ wt + qsec + am, data = mtcars, prior = R2(0.75)) SAMPLING FOR MODEL 'lm' NOW (CHAIN 1). Chain 1: Chain 1: Gradient evaluation took 0 seconds Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds. Chain 1: Adjust your expectations accordingly! Chain 1: Chain 1: Chain 1: Iteration: 1 / 2000 [ 0%] (Warmup) Chain 1: Iteration: 200 / 2000 [ 10%] (Warmup) Chain 1: Iteration: 400 / 2000 [ 20%] (Warmup) Chain 1: Iteration: 600 / 2000 [ 30%] (Warmup) Chain 1: Iteration: 800 / 2000 [ 40%] (Warmup) Chain 1: Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 1: Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 1: Iteration: 1200 / 2000 [ 60%] (Sampling) Chain 1: Iteration: 1400 / 2000 [ 70%] (Sampling) Chain 1: Iteration: 1600 / 2000 [ 80%] (Sampling) Chain 1: Iteration: 1800 / 2000 [ 90%] (Sampling) Chain 1: Iteration: 2000 / 2000 [100%] (Sampling) Chain 1: Chain 1: Elapsed Time: 3.998 seconds (Warm-up) Chain 1: 2.404 seconds (Sampling) Chain 1: 6.402 seconds (Total) Chain 1: SAMPLING FOR MODEL 'lm' NOW (CHAIN 2). Chain 2: Chain 2: Gradient evaluation took 0 seconds Chain 2: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds. Chain 2: Adjust your expectations accordingly! Chain 2: Chain 2: Chain 2: Iteration: 1 / 2000 [ 0%] (Warmup) Chain 2: Iteration: 200 / 2000 [ 10%] (Warmup) Chain 2: Iteration: 400 / 2000 [ 20%] (Warmup) Chain 2: Iteration: 600 / 2000 [ 30%] (Warmup) Chain 2: Iteration: 800 / 2000 [ 40%] (Warmup) Chain 2: Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 2: Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 2: Iteration: 1200 / 2000 [ 60%] (Sampling) Chain 2: Iteration: 1400 / 2000 [ 70%] (Sampling) Chain 2: Iteration: 1600 / 2000 [ 80%] (Sampling) Chain 2: Iteration: 1800 / 2000 [ 90%] (Sampling) Chain 2: Iteration: 2000 / 2000 [100%] (Sampling) Chain 2: Chain 2: Elapsed Time: 4.17 seconds (Warm-up) Chain 2: 5.554 seconds (Sampling) Chain 2: 9.724 seconds (Total) Chain 2: SAMPLING FOR MODEL 'lm' NOW (CHAIN 3). Chain 3: Chain 3: Gradient evaluation took 0 seconds Chain 3: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds. Chain 3: Adjust your expectations accordingly! Chain 3: Chain 3: Chain 3: Iteration: 1 / 2000 [ 0%] (Warmup) Chain 3: Iteration: 200 / 2000 [ 10%] (Warmup) Chain 3: Iteration: 400 / 2000 [ 20%] (Warmup) Chain 3: Iteration: 600 / 2000 [ 30%] (Warmup) Chain 3: Iteration: 800 / 2000 [ 40%] (Warmup) Chain 3: Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 3: Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 3: Iteration: 1200 / 2000 [ 60%] (Sampling) Chain 3: Iteration: 1400 / 2000 [ 70%] (Sampling) Chain 3: Iteration: 1600 / 2000 [ 80%] (Sampling) Chain 3: Iteration: 1800 / 2000 [ 90%] (Sampling) Chain 3: Iteration: 2000 / 2000 [100%] (Sampling) Chain 3: Chain 3: Elapsed Time: 3.83 seconds (Warm-up) Chain 3: 2.556 seconds (Sampling) Chain 3: 6.386 seconds (Total) Chain 3: SAMPLING FOR MODEL 'lm' NOW (CHAIN 4). Chain 4: Chain 4: Gradient evaluation took 0 seconds Chain 4: 1000 transitions using 10 leapfrog steps per transition would take 0 seconds. Chain 4: Adjust your expectations accordingly! Chain 4: Chain 4: Chain 4: Iteration: 1 / 2000 [ 0%] (Warmup) Chain 4: Iteration: 200 / 2000 [ 10%] (Warmup) Chain 4: Iteration: 400 / 2000 [ 20%] (Warmup) Chain 4: Iteration: 600 / 2000 [ 30%] (Warmup) Chain 4: Iteration: 800 / 2000 [ 40%] (Warmup) Chain 4: Iteration: 1000 / 2000 [ 50%] (Warmup) Chain 4: Iteration: 1001 / 2000 [ 50%] (Sampling) Chain 4: Iteration: 1200 / 2000 [ 60%] (Sampling) Chain 4: Iteration: 1400 / 2000 [ 70%] (Sampling) Chain 4: Iteration: 1600 / 2000 [ 80%] (Sampling) Chain 4: Iteration: 1800 / 2000 [ 90%] (Sampling) Chain 4: Iteration: 2000 / 2000 [100%] (Sampling) Chain 4: Chain 4: Elapsed Time: 4 seconds (Warm-up) Chain 4: 5.935 seconds (Sampling) Chain 4: 9.935 seconds (Total) Chain 4: Warning messages: 1: There were 13 divergent transitions after warmup. Increasing adapt_delta above 0.99 may help. See http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup 2: Examine the pairs() plot to diagnose sampling problems > plot(fit, prob = 0.8) > > # This tests that prophet is working > history <- data.frame(ds = seq(as.Date('2015-01-01'), as.Date('2016-01-01'), by = 'd'), + y = sin(1:366/200) + rnorm(366)/10) > m <- prophet(history) Disabling yearly seasonality. Run prophet with yearly.seasonality=TRUE to override this. Disabling daily seasonality. Run prophet with daily.seasonality=TRUE to override this. > future <- make_future_dataframe(m, periods = 365) > forecast <- predict(m, future) > plot(m, forecast) > > # This tests that CausalImpact is working > # First simulating some data > x1 <- 100 + arima.sim(model = list(ar = 0.999), n = 52) > y <- 1.2 * x1 + rnorm(52) > y[41:52] <- y[41:52] + 10 > data <- cbind(y, x1) > pre.period <- c(1, 40) > post.period <- c(41, 52) > # Then running CausalImpact > impact <- CausalImpact(data, pre.period, post.period) > plot(impact) Warning messages: 1: Removed 52 rows containing missing values (geom_path). 2: Removed 104 rows containing missing values (geom_path). -- | > ` — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://gist.github.com/cd84b63d13c8280ec91cdf576c8a0f52?email_source=notifications&email_token=AAHVUQFFGPKBXASFEUXDML3P5XNHLA5CNFSM4HOEUWY2YY3PNVWWK3TUL52HS4DFVNDWS43UINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAFUY4S#gistcomment-2961865>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHVUQFU4K3IHGPLMY2T2JTP5XNHLANCNFSM4HOEUWYQ> .

-- /Rasmus

Thank you Rasmus, my script is working with graphs plotting fine but i understand what am running can you please explain what we are trying to achieve even if there is a document to study.

@serifatf The only purpose of this script is to see if you have correctly installed the three packages, nothing else :)