Skip to content

Instantly share code, notes, and snippets.

@steveharoz

steveharoz/multiple comparison simulation.R

Last active December 1, 2022 02:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save steveharoz/2dc788c1d94c17805e53f0070ffd1b3b to your computer and use it in GitHub Desktop.
Save steveharoz/2dc788c1d94c17805e53f0070ffd1b3b to your computer and use it in GitHub Desktop.
Download ZIP
Simulate multiple comparisons to show that an adjustment is needed
Raw
multiple comparison simulation.R
COUNT = 100000
# How often does a single t-test of random data yield p<0.05?
replicate(COUNT,
t.test(rnorm(20))$p.value < 0.05
) %>% mean()
#> 0.05073
# 5% false positive rate
# How often will at least one of 10 t-tests of random data yield p<0.05?
replicate(COUNT,
replicate(10,
t.test(rnorm(20))$p.value < 0.05
) %>% any()
) %>% mean()
#> 0.4017
# 40% false positive rate
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment