Skip to content

Instantly share code, notes, and snippets.

Richard Morey richarddmorey

Block or report user

Report or block richarddmorey

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View post2.Rmd
The significance test is perhaps the most used statistical procedure in the world, though has never been without its detractors. This is the second of two posts exploring Neyman's frequentist arguments against the significance test; if you have not read Part 1, you should do so before continuing (["The frequentist case against the significance test, part 1"](http://bayesfactor.blogspot.co.uk/2015/03/the-frequentist-argument-against.html)).
<!--more-->
Neyman had two major arguments against the significance test:
1. **The significance test fails as an epistemic procedure.** There is no relationship between the $p$ value and rational belief. More broadly, the goal of statistical inference is tests with good error properties, not beliefs.
2. **The significance test fails as a test.** The lack of an alternative means that a significance test can yield arbitrary results.
View gist:4f808b875ab370cfd243
M = 10000 # Number of sims
power = .5 # power
## Binomial (fixed N)
N1 = rpois(M,2) + 1 ## random choice for N
X1 = rbinom(M,N1,power) ## Perform studies
mean(X1 / N1)
sd(X1 / N1)
View gist:cb22ee24f22a9257d04f
> sessionInfo()
R version 3.1.3 (2015-03-09)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.10.2 (Yosemite)
locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
@richarddmorey
richarddmorey / probit_utility.R
Last active Aug 29, 2015
Utility code for "What Are the Odds? Modern Relevance and Bayes Factor Solutions for MacAlister's Problem from the 1881 Educational Times."
View probit_utility.R
#####################
# Code for computing probit model Bayes factors for
# 2x2 contingency table data
# Code by Richard D. Morey, July, 2015
# For:
# "What Are the Odds? Modern Relevance and Bayes
# Factor Solutions for MacAlister's Problem from the 1881 Educational Times."
# Authors: Tahira Jamil, Maarten Marsman, Alexander Ly,
# Richard D. Morey, Eric-Jan Wagenmakers
#####################
@richarddmorey
richarddmorey / create_probit_plots.R
Last active Aug 29, 2015
Code to generate Bayes factors and plots for the probit models described in "What Are the Odds? Modern Relevance and Bayes Factor Solutions for MacAlister's Problem from the 1881 Educational Times."
View create_probit_plots.R
#####################
# Code for computing probit model Bayes factors for
# 2x2 contingency table data
# Data and plots
# Code by Richard D. Morey and EJ Wagenmakers, July, 2015
# For:
# "What Are the Odds? Modern Relevance and Bayes
# Factor Solutions for MacAlister's Problem from the 1881 Educational Times."
# Authors: Tahira Jamil, Maarten Marsman, Alexander Ly,
# Richard D. Morey, Eric-Jan Wagenmakers
@richarddmorey
richarddmorey / replication_CIs.R
Created Aug 30, 2015
Demonstration that inaccurate original results lead to poor replication coverage (even with no bias)
View replication_CIs.R
N1 = 10 # Participants in "original"; inaccurate
N2 = 200 # Participants in "replication"; accurate
M = 100000 # number of simulations
# CI setup
alpha = .05
zstar = qnorm(1-alpha/2)
@richarddmorey
richarddmorey / Neyman-etal-1969-whitetop.txt
Created Nov 12, 2015
Whitetop data from Neyman et al (1969) Table 2
View Neyman-etal-1969-whitetop.txt
A 11 64 68 -6 .66 .220 .305 -28 .15 .140 .206 -32 .12
B 20 75 75 -1 .93 .179 .257 -30 .051 .133 .192 -31 .070
C 33 83 80 4 .70 .167 .219 -24 .12 .139 .176 -21 .21
D 31 87 86 1 .96 .151 .208 -27 .050 .132 .179 -27 .072
E 31 85 77 -5 .49 .181 .189 -4 .77 .154 .169 -9 .56
F 48 91 98 -7 .079 .164 .195 -16 .25 .149 .191 -22 .12
Entire 174 95 99 -4 .24 .152 .184 -17 .20 .144 .182 -21 .13
@richarddmorey
richarddmorey / create-Neyman1969-plots.R
Last active Nov 12, 2015
R code to load Whitetop data from Neyman et al (1969) and recreate plots (Fig 2)
View create-Neyman1969-plots.R
url <- 'https://gist.githubusercontent.com/richarddmorey/862ca2681afd3cd85b3b/raw/8de4af8d99cbff8745f7b52fd9009410ff450bd5/Neyman-etal-1969-whitetop.txt'
library(RCurl)
df <- getURL(url, ssl.verifypeer=FALSE)
dat.all <- read.table(textConnection(df))
## Break into three data frames
dat.freq.wet = dat.all[,c(1:2,3:6)]
dat.rainfall.wet = dat.all[,c(1:2,7:10)]
dat.rainfall.all = dat.all[,c(1:2,11:14)]
View poster_figure.R
set.seed(6)
pdf(file="CI_Bayes_poster.pdf",width=8,height=4,ver="1.4")
layout(matrix(c(1,1,2,3),2,2))
par(mar=c(4,4.5,0,1),cex.lab=2,mgp=c(3,1,0))
ci.cols = c(rgb(1,0,0,1),rgb(0,0,0,1))
M = 30
View log_scale_demo.R
t = 2
N = 25
# Create vector of r values from one-fifth the default to five times
# the default; a *huge* range
r = exp(seq(-log(5),log(5),len=100)+log(sqrt(2)/2))
# Compute BF
bf = sapply(r, function(r) BayesFactor::ttest.tstat(t,N,rscale = r,simple=TRUE))
# Plots
par(mfrow=c(1,2),las=1)
You can’t perform that action at this time.