Mean reciprocal rank for Wiki Similarity Measures

mmr-wikisim.txt

WikiSim results

Article    See Also (unsortiert)    CoCit (sortiert)       CPA (sortiert)
----------------------------------------------------------------------
z          a, b, c, ,d               1: b, 2:c             1: d	
y          e, f                      1: f, 2: q, 3: x      1: p, 2: q, 3: a, 4: f
x          a, b                      1: b, 2: a            1: a, 2: b
----------------------------------------------------------------------


Berechnung - Mean reciprocal rank

query = article
results = sortierte Ergebnisse von CoCit/CPA
correct response = alle SeeAlso Links -> MRR Wert für jeden SeeAlso Link

CoCit für jeden Article
z:
+ a: 0
+ b: 1
+ c: 1/2
+ d: 0
==> MRRz = (0 + 1 + 1/2 + 0)/4 = 0.375 
y:
+ e: 0
+ f: 1
==> MRRy = (0 + 1)/2 = 0.5
==> MMR(CoCit) = (MMRz + MMRy)/2 = (0.375 + 0.5)/2 = 0.4375 
x:
+ a: 1/2
+ b: 1
==> MMRx = (1 + 1/2)/2 = 0.75 ######## ERROR / perfect result => score = 1



CPA
z:
+ a: 0
+ b: 0
+ c: 0
+ d: 1
==> MMRz = 1/4 
y:
+ e: 0
+ f: 1/4
==> MMRy = (1/4)/2 = 0.125
==> MMR(CPA) =  (MMRz + MMRy)/2 = (0.25 + 0.125)/2 = 0.1875 


-------

MMR(CoCit) > MMR(CPA): Also CoCit nach Mean reciprocal rank besser als CPA?

####

100% match - score = 1
50% match  
- SeeAlso A, B -- CPA A, C --- score = 1/2
- SeeAlso A, B -- CPA C, A --- score = 1/2

0% match   - score = 0