View post2.Rmd
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The significance test is perhaps the most used statistical procedure in the world, though has never been without its detractors. This is the second of two posts exploring Neyman's frequentist arguments against the significance test; if you have not read Part 1, you should do so before continuing (["The frequentist case against the significance test, part 1"](http://bayesfactor.blogspot.co.uk/2015/03/the-frequentist-argument-against.html)). | |
<!--more--> | |
Neyman had two major arguments against the significance test: | |
1. **The significance test fails as an epistemic procedure.** There is no relationship between the $p$ value and rational belief. More broadly, the goal of statistical inference is tests with good error properties, not beliefs. | |
2. **The significance test fails as a test.** The lack of an alternative means that a significance test can yield arbitrary results. |
View gist:4f808b875ab370cfd243
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
M = 10000 # Number of sims | |
power = .5 # power | |
## Binomial (fixed N) | |
N1 = rpois(M,2) + 1 ## random choice for N | |
X1 = rbinom(M,N1,power) ## Perform studies | |
mean(X1 / N1) | |
sd(X1 / N1) |
View gist:cb22ee24f22a9257d04f
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
> sessionInfo() | |
R version 3.1.3 (2015-03-09) | |
Platform: x86_64-apple-darwin13.4.0 (64-bit) | |
Running under: OS X 10.10.2 (Yosemite) | |
locale: | |
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8 | |
attached base packages: | |
[1] stats graphics grDevices utils datasets methods base |
View probit_utility.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
##################### | |
# Code for computing probit model Bayes factors for | |
# 2x2 contingency table data | |
# Code by Richard D. Morey, July, 2015 | |
# For: | |
# "What Are the Odds? Modern Relevance and Bayes | |
# Factor Solutions for MacAlister's Problem from the 1881 Educational Times." | |
# Authors: Tahira Jamil, Maarten Marsman, Alexander Ly, | |
# Richard D. Morey, Eric-Jan Wagenmakers | |
##################### |
View create_probit_plots.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
##################### | |
# Code for computing probit model Bayes factors for | |
# 2x2 contingency table data | |
# Data and plots | |
# Code by Richard D. Morey and EJ Wagenmakers, July, 2015 | |
# For: | |
# "What Are the Odds? Modern Relevance and Bayes | |
# Factor Solutions for MacAlister's Problem from the 1881 Educational Times." | |
# Authors: Tahira Jamil, Maarten Marsman, Alexander Ly, | |
# Richard D. Morey, Eric-Jan Wagenmakers |
View replication_CIs.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
N1 = 10 # Participants in "original"; inaccurate | |
N2 = 200 # Participants in "replication"; accurate | |
M = 100000 # number of simulations | |
# CI setup | |
alpha = .05 | |
zstar = qnorm(1-alpha/2) |
View Neyman-etal-1969-whitetop.txt
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A 11 64 68 -6 .66 .220 .305 -28 .15 .140 .206 -32 .12 | |
B 20 75 75 -1 .93 .179 .257 -30 .051 .133 .192 -31 .070 | |
C 33 83 80 4 .70 .167 .219 -24 .12 .139 .176 -21 .21 | |
D 31 87 86 1 .96 .151 .208 -27 .050 .132 .179 -27 .072 | |
E 31 85 77 -5 .49 .181 .189 -4 .77 .154 .169 -9 .56 | |
F 48 91 98 -7 .079 .164 .195 -16 .25 .149 .191 -22 .12 | |
Entire 174 95 99 -4 .24 .152 .184 -17 .20 .144 .182 -21 .13 |
View create-Neyman1969-plots.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
url <- 'https://gist.githubusercontent.com/richarddmorey/862ca2681afd3cd85b3b/raw/8de4af8d99cbff8745f7b52fd9009410ff450bd5/Neyman-etal-1969-whitetop.txt' | |
library(RCurl) | |
df <- getURL(url, ssl.verifypeer=FALSE) | |
dat.all <- read.table(textConnection(df)) | |
## Break into three data frames | |
dat.freq.wet = dat.all[,c(1:2,3:6)] | |
dat.rainfall.wet = dat.all[,c(1:2,7:10)] | |
dat.rainfall.all = dat.all[,c(1:2,11:14)] |
View poster_figure.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
set.seed(6) | |
pdf(file="CI_Bayes_poster.pdf",width=8,height=4,ver="1.4") | |
layout(matrix(c(1,1,2,3),2,2)) | |
par(mar=c(4,4.5,0,1),cex.lab=2,mgp=c(3,1,0)) | |
ci.cols = c(rgb(1,0,0,1),rgb(0,0,0,1)) | |
M = 30 |
View log_scale_demo.R
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
t = 2 | |
N = 25 | |
# Create vector of r values from one-fifth the default to five times | |
# the default; a *huge* range | |
r = exp(seq(-log(5),log(5),len=100)+log(sqrt(2)/2)) | |
# Compute BF | |
bf = sapply(r, function(r) BayesFactor::ttest.tstat(t,N,rscale = r,simple=TRUE)) | |
# Plots | |
par(mfrow=c(1,2),las=1) |
OlderNewer