Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
A Significantly Improved Significance Test! Not! (More context in this blogpost: http://www.sumsar.net/blog/2014/02/a-significantly-improved-test/)
# Test of Significance, takes the same arguments as t.test() .
signif.test <- function(x, ...) {
p <- t.test(x, ...)$p.value
# List of p excuses retrieved from http://mchankins.wordpress.com/2013/04/21/still-not-significant-2/
p_excuses <- c(
"(barely) not statistically significant <p>",
"a barely detectable statistically significant difference <p>",
"a borderline significant trend <p>",
"a certain trend toward significance <p>",
"a clear tendency to significance <p>",
"a clear trend <p>",
"a clear, strong trend <p>",
"a considerable trend toward significance <p>",
"a decreasing trend <p>",
"a definite trend <p>",
"a distinct trend toward significance <p>",
"a favorable trend <p>",
"a favourable statistical trend <p>",
"a little significant <p>",
"a margin at the edge of significance <p>",
"a marginal trend <p>",
"a marginal trend toward significance <p>",
"a marked trend <p>",
"a mild trend <p>",
"a moderate trend toward significance <p>",
"a near-significant trend <p>",
"a negative trend <p>",
"a nonsignificant trend <p>",
"a nonsignificant trend toward significance <p>",
"a notable trend <p>",
"a numerical increasing trend <p>",
"a numerical trend <p>",
"a positive trend <p>",
"a possible trend <p>",
"a possible trend toward significance <p>",
"a pronounced trend <p>",
"a reliable trend <p>",
"a robust trend toward significance <p>",
"a significant trend <p>",
"a slight slide towards significance <p>",
"a slight tendency toward significance<p>",
"a slight trend <p>",
"a slight trend toward significance <p>",
"a slightly increasing trend <p>",
"a small trend <p>",
"a statistical trend <p>",
"a statistical trend toward significance <p>",
"a strong tendency towards statistical significance <p>",
"a strong trend <p>",
"a strong trend toward significance <p>",
"a substantial trend toward significance <p>",
"a suggestive trend <p>",
"a trend close to significance <p>",
"a trend significance level <p>",
"a trend that approached significance <p>",
"a very slight trend toward significance <p>",
"a weak trend <p>",
"a weak trend toward significance <p>",
"a worrying trend <p>",
"all but significant <p>",
"almost achieved significance <p>",
"almost approached significance <p>",
"almost attained significance <p>",
"almost became significant <p>",
"almost but not quite significant <p>",
"almost clinically significant <p>",
"almost insignificant <p>",
"almost marginally significant <p>",
"almost non-significant <p>",
"almost reached statistical significance <p>",
"almost significant <p>",
"almost significant tendency <p>",
"almost statistically significant <p>",
"an adverse trend <p>",
"an apparent trend <p>",
"an associative trend <p>",
"an elevated trend <p>",
"an encouraging trend <p>",
"an established trend <p>",
"an evident trend <p>",
"an expected trend <p>",
"an important trend <p>",
"an increasing trend <p>",
"an interesting trend <p>",
"an inverse trend toward significance <p>",
"an observed trend <p>",
"an obvious trend <p>",
"an overall trend <p>",
"an unexpected trend <p>",
"an unexplained trend <p>",
"an unfavorable trend <p>",
"appeared to be marginally significant <p>",
"approached acceptable levels of statistical significance <p>",
"approached but did not quite achieve significance <p>",
"approached but fell short of significance <p>",
"approached conventional levels of significance <p>",
"approached near significance <p>",
"approached our criterion of significance <p>",
"approached significant <p>",
"approached the borderline of significance <p>",
"approached the level of significance <p>",
"approached trend levels of significance <p>",
"approached, but did reach, significance <p>",
"approaches but fails to achieve a customary level of statistical significance <p>",
"approaches statistical significance <p>",
"approaching a level of significance <p>",
"approaching an acceptable significance level <p>",
"approaching borderline significance <p>",
"approaching borderline statistical significance <p>",
"approaching but not reaching significance <p>",
"approaching clinical significance <p>",
"approaching close to significance <p>",
"approaching conventional significance levels <p>",
"approaching conventional statistical significance <p>",
"approaching formal significance <p>",
"approaching independent prognostic significance <p>",
"approaching marginal levels of significance <p>",
"approaching marginal significance <p>",
"approaching more closely significance <p>",
"approaching our preset significance level <p>",
"approaching prognostic significance <p>",
"approaching significance <p>",
"approaching the traditional significance level <p>",
"approaching to statistical significance <p>",
"approaching, although not reaching, significance <p>",
"approaching, but not reaching, significance <p>",
"approximately significant <p>",
"approximating significance <p>",
"arguably significant <p>",
"as good as significant <p>",
"at the brink of significance <p>",
"at the cusp of significance <p>",
"at the edge of significance <p>",
"at the limit of significance <p>",
"at the limits of significance <p>",
"at the margin of significance <p>",
"at the margin of statistical significance <p>",
"at the verge of significance <p>",
"at the very edge of significance <p>",
"barely below the level of significance <p>",
"barely escaped statistical significance <p>",
"barely escapes being statistically significant at the 5% risk level <p>",
"barely failed to attain statistical significance <p>",
"barely fails to attain statistical significance at conventional levels <p>",
"barely insignificant <p>",
"barely missed statistical significance <p>",
"barely missed the commonly acceptable significance level <p>",
"barely outside the range of significance <p>",
"barely significant <p>",
"below (but verging on) the statistical significant level <p>",
"better trends of improvement <p>",
"bordered on a statistically significant value <p>",
"bordered on being significant <p>",
"bordered on being statistically significant <p>",
"bordered on but was not less than the accepted level of significance <p>",
"bordered on significant <p>",
"borderline conventional significance <p>",
"borderline level of statistical significance <p>",
"borderline significant <p>",
"borderline significant trends <p>",
"close to a marginally significant level <p>",
"close to being significant <p>",
"close to being statistically significant <p>",
"close to borderline significance <p>",
"close to the boundary of significance <p>",
"close to the level of significance <p>",
"close to the limit of significance <p>",
"close to the margin of significance <p>",
"close to the margin of statistical significance <p>",
"closely approaches the brink of significance <p>",
"closely approaches the statistical significance <p>",
"closely approximating significance <p>",
"closely not significant <p>",
"closely significant <p>",
"close-to-significant <p>",
"did not achieve conventional threshold levels of statistical significance <p>",
"did not exceed the conventional level of statistical significance <p>",
"did not quite achieve acceptable levels of statistical significance <p>",
"did not quite achieve significance <p>",
"did not quite achieve the conventional levels of significance <p>",
"did not quite achieve the threshold for statistical significance <p>",
"did not quite attain conventional levels of significance <p>",
"did not quite reach a statistically significant level <p>",
"did not quite reach conventional levels of statistical significance <p>",
"did not quite reach statistical significance <p>",
"did not reach the traditional level of significance <p>",
"did not reach the usually accepted level of clinical significance <p>",
"difference was apparent <p>",
"direction heading towards significance <p>",
"does not appear to be sufficiently significant <p>",
"does not narrowly reach statistical significance <p>",
"does not reach the conventional significance level <p>",
"effectively significant <p>",
"equivocal significance <p>",
"essentially significant <p>",
"extremely close to significance <p>",
"failed to reach significance on this occasion <p>",
"failed to reach statistical significance <p>",
"fairly close to significance <p>",
"fairly significant <p>",
"falls just short of standard levels of statistical significance <p>",
"fell (just) short of significance <p>",
"fell barely short of significance <p>",
"fell just short of significance <p>",
"fell just short of statistical significance <p>",
"fell just short of the traditional definition of statistical significance <p>",
"fell marginally short of significance <p>",
"fell narrowly short of significance <p>",
"fell only marginally short of significance <p>",
"fell only short of significance <p>",
"fell short of significance <p>",
"fell slightly short of significance <p>",
"fell somewhat short of significance <p>",
"felt short of significance <p>",
"flirting with conventional levels of significance <p>",
"heading towards significance <p>",
"highly significant <p>",
"hint of significance <p>",
"hovered around significance <p>",
"hovered at nearly a significant level <p>",
"hovering closer to statistical significance <p>",
"hovers on the brink of significance <p>",
"in the edge of significance <p>",
"in the verge of significance <p>",
"inconclusively significant <p>",
"indeterminate significance <p>",
"indicative significance <p>",
"is just outside the conventional levels of significance <p>",
"just about significant <p>",
"just above the arbitrary level of significance <p>",
"just above the margin of significance <p>",
"just at the conventional level of significance <p>",
"just barely below the level of significance <p>",
"just barely failed to reach significance <p>",
"just barely insignificant <p>",
"just barely statistically significant <p>",
"just beyond significance <p>",
"just borderline significant <p>",
"just escaped significance <p>",
"just failed significance <p>",
"just failed to be significant <p>",
"just failed to reach statistical significance <p>",
"just failing to reach statistical significance <p>",
"just fails to reach conventional levels of statistical significance <p>",
"just lacked significance <p>",
"just marginally significant <p>",
"just missed being statistically significant <p>",
"just missing significance <p>",
"just on the verge of significance <p>",
"just outside accepted levels of significance <p>",
"just outside levels of significance <p>",
"just outside the bounds of significance <p>",
"just outside the conventional levels of significance <p>",
"just outside the level of significance <p>",
"just outside the limits of significance <p>",
"just outside the traditional bounds of significance <p>",
"just over the limits of statistical significance <p>",
"just short of significance <p>",
"just shy of significance <p>",
"just skirting the boundary of significance <p>",
"just tendentially significant <p>",
"just very slightly missed the significance level <p>",
"leaning towards significance <p>",
"leaning towards statistical significance <p>",
"likely to be significant <p>",
"loosely significant <p>",
"marginal significance <p>",
"marginally and negatively significant <p>",
"marginally insignificant <p>",
"marginally nonsignificant <p>",
"marginally outside the level of significance <p>",
"marginally significant <p>",
"marginally significant tendency <p>",
"marginally statistically significant <p>",
"may not be significant <p>",
"medium level of significance <p>",
"mildly significant <p>",
"missed narrowly statistical significance <p>",
"moderately significant <p>",
"modestly significant <p>",
"narrowly avoided significance <p>",
"narrowly eluded statistical significance <p>",
"narrowly escaped significance <p>",
"narrowly evaded statistical significance <p>",
"narrowly failed significance <p>",
"narrowly missed achieving significance <p>",
"narrowly missed overall significance <p>",
"narrowly missed significance <p>",
"narrowly missed standard significance levels <p>",
"narrowly missed the significance level <p>",
"narrowly missing conventional significance <p>",
"near limit significance <p>",
"near miss of statistical significance <p>",
"near nominal significance <p>",
"near significance <p>",
"near to statistical significance <p>",
"near/possible significance<p>",
"near-borderline significance <p>",
"near-certain significance <p>",
"nearing significance <p>",
"nearly acceptable level of significance <p>",
"nearly approaches statistical significance <p>",
"nearly borderline significance <p>",
"nearly negatively significant <p>",
"nearly positively significant <p>",
"nearly reached a significant level <p>",
"nearly reaching the level of significance <p>",
"nearly significant <p>",
"nearly significant tendency <p>",
"nearly, but not quite significant <p>",
"near-marginal significance <p>",
"near-significant <p>",
"near-to-significance <p>",
"near-trend significance <p>",
"nominally significant <p>",
"non-insignificant result <p>",
"non-significant in the statistical sense <p>",
"not absolutely significant but very probably so <p>",
"not as significant <p>",
"not clearly significant <p>",
"not completely significant <p>",
"not completely statistically significant <p>",
"not conventionally significant <p>, but..",
"not currently significant <p>",
"not decisively significant <p>",
"not entirely significant <p>",
"not especially significant <p>",
"not exactly significant <p>",
"not extremely significant <p>",
"not formally significant <p>",
"not fully significant <p>",
"not globally significant <p>",
"not highly significant <p>",
"not insignificant <p>",
"not markedly significant <p>",
"not moderately significant <p>",
"not non-significant <p>",
"not numerically significant <p>",
"not obviously significant <p>",
"not overly significant <p>",
"not quite borderline significance <p>",
"not quite reach the level of significance <p>",
"not quite significant <p>",
"not quite within the conventional bounds of statistical significance <p>",
"not reliably significant <p>",
"not remarkably significant <p>",
"not significant by common standards <p>",
"not significant by conventional standards <p>",
"not significant by traditional standards <p>",
"not significant in the formal statistical sense <p>",
"not significant in the narrow sense of the word <p>",
"not significant in the normally accepted statistical sense <p>",
"not significantly significant but..clinically meaningful <p>",
"not statistically quite significant <p>",
"not strictly significant <p>",
"not strictly speaking significant <p>",
"not technically significant <p>",
"not that significant <p>",
"not to an extent that was fully statistically significant <p>",
"not totally significant <p>",
"not unequivocally significant <p>",
"not very definitely significant <p>",
"not very definitely significant from the statistical point of view <p>",
"not very far from significance <p>",
"not very significant <p>",
"not very statistically significant <p>",
"not wholly significant <p>",
"not yet significant <p>",
"not strongly significant <p>",
"noticeably significant <p>",
"on the border of significance <p>",
"on the borderline of significance <p>",
"on the borderlines of significance <p>",
"on the boundaries of significance <p>",
"on the boundary of significance <p>",
"on the brink of significance <p>",
"on the cusp of conventional statistical significance <p>",
"on the cusp of significance <p>",
"on the edge of significance <p>",
"on the limit to significant <p>",
"on the margin of significance <p>",
"on the threshold of significance <p>",
"on the verge of significance <p>",
"on the very borderline of significance <p>",
"on the very fringes of significance <p>",
"on the very limits of significance <p>",
"only a little short of significance <p>",
"only just failed to meet statistical significance <p>",
"only just insignificant <p>",
"only marginally fails to be significant at the 95% level <p>",
"only marginally nearly insignificant <p>",
"only marginally significant <p>",
"only slightly less than significant <p>",
"only slightly missed the conventional threshold of significance <p>",
"only slightly missed the level of significance <p>",
"only slightly missed the significance level <p>",
"only slightly non-significant <p>",
"only slightly significant <p>",
"partial significance <p>",
"partially significant <p>",
"partly significant <p>",
"perceivable statistical significance <p>",
"possible significance <p>",
"possibly marginally significant <p>",
"possibly significant <p>",
"possibly statistically significant <p>",
"potentially significant <p>",
"practically significant <p>",
"probably not experimentally significant <p>",
"probably not significant <p>",
"probably not statistically significant <p>",
"probably significant <p>",
"provisionally significant <p>",
"quasi-significant <p>",
"questionably significant <p>",
"quite close to significance at the 10% level <p>",
"quite significant <p>",
"rather marginal significance <p>",
"reached borderline significance <p>",
"reached near significance <p>",
"reasonably significant <p>",
"remarkably close to significance <p>",
"resides on the edge of significance <p>",
"roughly significant <p>",
"scarcely significant <p>",
"significant tendency <p>",
"significant to some degree <p>",
"significant, or close to significant effects <p>",
"significantly better overall <p>",
"significantly significant <p>",
"similar but not nonsignificant trends <p>",
"slight evidence of significance <p>",
"slight non-significance <p>",
"slight significance <p>",
"slight tendency toward significance <p>",
"slightly above the level of significance <p>",
"slightly below the level of significance <p>",
"slightly exceeded significance level <p>",
"slightly failed to reach statistical significance <p>",
"slightly insignificant <p>",
"slightly less than needed for significance <p>",
"slightly marginally significant <p>",
"slightly missed being of statistical significance <p>",
"slightly missed statistical significance <p>",
"slightly missed the conventional level of significance <p>",
"slightly missed the level of statistical significance <p>",
"slightly missed the margin of significance <p>",
"slightly not significant <p>",
"slightly outside conventional statistical significance <p>",
"slightly outside the margins of significance <p>",
"slightly outside the range of significance <p>",
"slightly outside the significance level <p>",
"slightly outside the statistical significance level <p>",
"slightly significant <p>",
"somewhat marginally significant <p>",
"somewhat short of significance <p>",
"somewhat significant <p>",
"somewhat statistically significant <p>",
"strong trend toward significance <p>",
"sufficiently close to significance <p>",
"suggestive but not quite significant <p>",
"suggestive of a significant trend <p>",
"suggestive of statistical significance <p>",
"suggestively significant <p>",
"tailed to insignificance <p>",
"tantalisingly close to significance <p>",
"technically not significant <p>",
"teetering on the brink of significance <p>",
"tend to significant <p>",
"tended to approach significance <p>",
"tended to be significant <p>",
"tended toward significance <p>",
"tendency toward significance <p>",
"tendency toward statistical significance <p>",
"tends to approach significance <p>",
"tentatively significant <p>",
"too far from significance <p>",
"trend bordering on statistical significance <p>",
"trend in a significant direction <p>",
"trend in the direction of significance <p>",
"trend significance level <p>",
"trend toward <p>",
"trending towards significance <p>",
"trending towards significant <p>",
"uncertain significance <p>",
"vaguely significant <p>",
"verged on being significant <p>",
"verging on significance <p>",
"verging on the statistically significant <p>",
"verging-on-significant <p>",
"very close to approaching significance <p>",
"very close to significant <p>",
"very close to the conventional level of significance <p>",
"very close to the cut-off for significance <p>",
"very close to the established statistical significance level of p=0.05 <p>",
"very close to the threshold of significance <p>",
"very closely approaches the conventional significance level <p>",
"very closely brushed the limit of statistical significance <p>",
"very narrowly missed significance <p>",
"very nearly significant <p>",
"very slightly non-significant <p>",
"very slightly significant <p>",
"virtually significant <p>",
"weak significance <p>",
"weakened..significance <p>",
"weakly non-significant <p>",
"weakly significant <p>",
"weakly statistically significant <p>",
"well-nigh significant <p>")
cat(" Test of Significance\n")
cat("\n")
cat("H0: p will be more than 0.05\n")
cat("H1: p will be less than 0.05\n")
cat("\n")
cat(" Result\n")
if(p < 0.0001) {
cat("significant beyond doubt (p < 0.0001)\n")
} else if(p < 0.001) {
cat("extremely significant (p < 0.001)\n")
} else if(p < 0.01) {
cat(paste("highly significant (p = ", round(p, 4) , ")\n", sep="" ))
} else if(p < 0.05) {
cat(paste("significant (p = ", round(p, 4) , ")\n", sep="" ))
} else if(p < 0.12) {
excuse <- gsub("<p>", paste("(p = ", round(p, 4), ")",sep=""), p_excuses[sample(seq_along(p_excuses), 1)])
cat(excuse, "\n")
} else {
cat("not significant (n.s.)")
}
invisible(p)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment