Skip to content

Instantly share code, notes, and snippets.

@instance01
Created July 18, 2020 11:30
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save instance01/badf4a318c22d614500088573332d553 to your computer and use it in GitHub Desktop.
Save instance01/badf4a318c22d614500088573332d553 to your computer and use it in GitHub Desktop.
import numpy as np
def calc_cohens_kappa(matrix):
p0 = (matrix[0][0] + matrix[1][1]) / matrix.sum()
pyes = (matrix[0].sum() / matrix.sum()) * ((matrix[0][0] + matrix[1][0]) / matrix.sum())
pno = (matrix[1].sum() / matrix.sum()) * ((matrix[0][1] + matrix[1][1]) / matrix.sum())
pe = pyes + pno
k = (p0 - pe) / (1 - pe)
print(k)
calc_cohens_kappa(np.array([[20, 5], [10, 15]]))
calc_cohens_kappa(np.array([[45, 15], [25, 15]]))
calc_cohens_kappa(np.array([[25, 35], [5, 35]]))
calc_cohens_kappa(np.array([[37, 14], [17, 32]]))
calc_cohens_kappa(np.array([[65, 8], [7, 20]]))
calc_cohens_kappa(np.array([[90, 4], [5, 1]]))
"""
κ < 0 poor agreement
0 < κ < 0.2 slight agreement
0.21 < κ < 0.4 fair agreement
0.41 < κ < 0.6 moderate agreement
0.61 < κ < 0.8 substantial agreement
0.81 < κ < 1 (almost) perfect agreement
"""
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment