MZ : the Match Zoo evaluation run on my machine Mine: my evaluation script run on my machine
ANMM
MZ:
map=0.610744
ndcg@1=0.459916
ndcg@3=0.603051
ndcg@5=0.646496
ndcg@10=0.698186
ndcg@20=0.706716
Mine:
map : 0.5940871282334572
ndcg@ 1 : 0.459915611814346
ndcg@ 3 : 0.607270198545944
ndcg@ 5 : 0.650162952846763
ndcg@ 10 : 0.701300782797327
ndcg@ 20 : 0.7114128188841645
==================================
MVLSTM:
MZ:
map=0.626253
ndcg@1=0.497890
ndcg@3=0.608242
ndcg@5=0.671299
ndcg@10=0.710842
ndcg@20=0.719987
Mine:
map : 0.6029693307776052
ndcg@ 1 : 0.4978902953586498
ndcg@ 3 : 0.6082418119892877
ndcg@ 5 : 0.6712989992819381
ndcg@ 10 : 0.7108418546306594
ndcg@ 20 : 0.7199866210065858
=====================================
DRMM
MZ:
map=0.604482
ndcg@1=0.447257
ndcg@3=0.602549
ndcg@5=0.648429
ndcg@10=0.689186
ndcg@20=0.704016
Mine:
map : 0.5894358178350794
ndcg@ 1 : 0.4472573839662447
ndcg@ 3 : 0.6067680114452771
ndcg@ 5 : 0.6520963714993983
ndcg@ 10 : 0.6923006011390016
ndcg@ 20 : 0.7087128212846961
====================================
ARCI
MZ:
map=0.586587
ndcg@1=0.443038
ndcg@3=0.559602
ndcg@5=0.630911
ndcg@10=0.669301
ndcg@20=0.686561
Mine:
map : 0.5655101582835583
ndcg@ 1 : 0.4430379746835443
ndcg@ 3 : 0.5596017895812407
ndcg@ 5 : 0.6309109425437026
ndcg@ 10 : 0.6693013960049794
ndcg@ 20 : 0.6865609825369599
=====================================
ARCII
MZ:
map=0.382256
ndcg@1=0.198312
ndcg@3=0.293003
ndcg@5=0.416683
ndcg@10=0.502918
ndcg@20=0.513140
Mine:
Not Available as it isn't working
======================================
MatchPyramid
MZ:
map=0.642877
ndcg@1=0.476793
ndcg@3=0.640863
ndcg@5=0.685916
ndcg@10=0.725052
ndcg@20=0.732838
Mine:
map : 0.6204574621808965
ndcg@ 1 : 0.4767932489451477
ndcg@ 3 : 0.6429727765203649
ndcg@ 5 : 0.6862080716093452
ndcg@ 10 : 0.7253449354455153
ndcg@ 20 : 0.7331301838098461
===================================
DUET
MZ:
map=0.630124
ndcg@1=0.472574
ndcg@3=0.625243
ndcg@5=0.676017
ndcg@10=0.711811
ndcg@20=0.723448
Mine:
map : 0.6076386125250608
ndcg@ 1 : 0.47257383966244726
ndcg@ 3 : 0.6252427774109915
ndcg@ 5 : 0.6760173358856317
ndcg@ 10 : 0.7118107175775565
ndcg@ 20 : 0.7234479952474052
======================================
DRMM TKS
MZ:
map=0.647169
ndcg@1=0.506329
ndcg@3=0.639266
ndcg@5=0.686918
ndcg@10=0.725678
ndcg@20=0.734016
Mine:
map : 0.6317413166410893
ndcg@ 1 : 0.5063291139240507
ndcg@ 3 : 0.6434852886326535
ndcg@ 5 : 0.6905851544372844
ndcg@ 10 : 0.7287928253446959
ndcg@ 20 : 0.7387126758837185
==================================
CDSSM
map=0.419511
ndcg@1=0.236287
ndcg@3=0.381432
ndcg@5=0.449866
ndcg@10=0.519577
ndcg@20=0.556570
map : 0.4019769779348184
ndcg@ 1 : 0.23628691983122363
ndcg@ 3 : 0.38354137694929374
ndcg@ 5 : 0.4501585936573311
ndcg@ 10 : 0.5198697519495901
ndcg@ 20 : 0.5568626793778719
=================================
DSSM
MZ:
map=0.562450
ndcg@1=0.396624
ndcg@3=0.540033
ndcg@5=0.608210
ndcg@10=0.656986
ndcg@20=0.670303
Mine:
map : 0.5436485818910034
ndcg@ 1 : 0.39662447257383965
ndcg@ 3 : 0.5400325692705683
ndcg@ 5 : 0.6083950014480743
ndcg@ 10 : 0.6571710846883538
ndcg@ 20 : 0.6704883504031652
===============================
w2v averaging 50 dim
map : 0.525039852586227
ndcg@ 1 : 0.37844036697247707
ndcg@ 3 : 0.5288832435950492
ndcg@ 5 : 0.5938376161901073
ndcg@ 10 : 0.6426858123695702
ndcg@ 20 : 0.6578442782634695
===============================
d2v :
map : 0.3694044782081827
ndcg@ 1 : 0.1908713692946058
ndcg@ 3 : 0.33425007107376825
ndcg@ 5 : 0.4149963330213527
ndcg@ 10 : 0.49378684260187083
ndcg@ 20 : 0.5293908121349118
==============================