Skip to content

Instantly share code, notes, and snippets.

@OsmanMutlu
Created October 6, 2018 08:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save OsmanMutlu/02541286614333de1d0136a70cfd6caa to your computer and use it in GitHub Desktop.
Save OsmanMutlu/02541286614333de1d0136a70cfd6caa to your computer and use it in GitHub Desktop.
Krippendorff's alpha for two annotators with nominal data
import pandas as pd
def calculate_Kalpha2(y1,y2):
df = pd.concat([y1,y2], axis=1)
df.columns = ['y1', 'y2']
n = 2*len(df)
agg = 0
all = 0
for label in df.y1.unique().tolist():
agg += 2*len(df[(df.y1 == label) & (df.y2 == label)])
asd = len(df[df.y1 == label]) + len(df[df.y2 == label])
all += asd*(asd - 1)
nom = (n-1)*agg - all
denom = n*(n - 1) - all
return nom/denom
#y1 and y2 are series. Annotations of two different annotators
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment