Skip to content

Instantly share code, notes, and snippets.

@dvdblk

dvdblk/crf_eval.txt

Created December 12, 2019 17:11
Show Gist options
  • Save dvdblk/7f93bbab205ac2847ea1396aa31086d0 to your computer and use it in GitHub Desktop.
Save dvdblk/7f93bbab205ac2847ea1396aa31086d0 to your computer and use it in GitHub Desktop.
Download ZIP
ML4NLP Exercise 04 - Model Evaluation
Raw
crf_eval.txt
STRICT: Found: 1162 outer and 0 inner phrases; Gold: 6177 (outer) and 0 (inner).
LOOSE: Found: 1162 outer and 0 inner phrases; Gold: 6177 (outer) and 0 (inner).
1. Strict, Combined Evaluation (official):
Accuracy: 95.75%;
Precision: 67.81%;
Recall: 12.76%;
FB1: 21.47
2. Loose, Combined Evaluation:
Accuracy: 95.76%;
Precision: 68.24%;
Recall: 12.84%;
FB1: 21.61
3.1 Per-Level Evaluation (outer chunks):
Accuracy: 91.50%;
Precision: 67.81%;
Recall: 12.76%;
FB1: 21.47
3.2 Per-Level Global Evaluation (inner chunks):
Accuracy: 100.00%;
Precision: 0.00%;
Recall: 0.00%;
FB1: 0.00
Evaluation per type and mode:
=============================
==> LOC
==============
Outer strict: Precision: 73.40%; Recall: 12.13%; FB1: 20.82
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 82.96%; Recall: 21.72%; FB1: 34.42
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> LOCderiv
==============
Outer strict: Precision: 89.94%; Recall: 54.19%; FB1: 67.63
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> LOCpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORG
==============
Outer strict: Precision: 50.00%; Recall: 3.91%; FB1: 7.26
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 50.55%; Recall: 3.46%; FB1: 6.47
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORGderiv
==============
Outer strict: Precision: 100.00%; Recall: 12.50%; FB1: 22.22
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORGpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTH
==============
Outer strict: Precision: 40.00%; Recall: 0.86%; FB1: 1.69
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 62.50%; Recall: 3.21%; FB1: 6.11
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTHderiv
==============
Outer strict: Precision: 78.95%; Recall: 38.46%; FB1: 51.72
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTHpart
==============
Outer strict: Precision: 66.67%; Recall: 9.52%; FB1: 16.67
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PER
==============
Outer strict: Precision: 50.49%; Recall: 12.58%; FB1: 20.14
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 50.37%; Recall: 12.17%; FB1: 19.60
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PERderiv
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PERpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Raw
rnn_eval.txt
STRICT: Found: 1766 outer and 0 inner phrases; Gold: 6137 (outer) and 0 (inner).
LOOSE: Found: 1766 outer and 0 inner phrases; Gold: 6137 (outer) and 0 (inner).
1. Strict, Combined Evaluation (official):
Accuracy: 95.75%;
Precision: 46.15%;
Recall: 13.28%;
FB1: 20.63
2. Loose, Combined Evaluation:
Accuracy: 95.76%;
Precision: 46.49%;
Recall: 13.38%;
FB1: 20.78
3.1 Per-Level Evaluation (outer chunks):
Accuracy: 91.50%;
Precision: 46.15%;
Recall: 13.28%;
FB1: 20.63
3.2 Per-Level Global Evaluation (inner chunks):
Accuracy: 100.00%;
Precision: 0.00%;
Recall: 0.00%;
FB1: 0.00
Evaluation per type and mode:
=============================
==> LOC
==============
Outer strict: Precision: 59.39%; Recall: 18.31%; FB1: 27.99
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 72.22%; Recall: 26.09%; FB1: 38.33
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> LOCderiv
==============
Outer strict: Precision: 90.63%; Recall: 53.67%; FB1: 67.42
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> LOCpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORG
==============
Outer strict: Precision: 7.77%; Recall: 2.72%; FB1: 4.03
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 7.77%; Recall: 2.35%; FB1: 3.61
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORGderiv
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> ORGpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTH
==============
Outer strict: Precision: 1.82%; Recall: 0.14%; FB1: 0.27
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 19.18%; Recall: 1.81%; FB1: 3.30
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTHderiv
==============
Outer strict: Precision: 72.22%; Recall: 33.33%; FB1: 45.61
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> OTHpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PER
==============
Outer strict: Precision: 36.28%; Recall: 9.83%; FB1: 15.47
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Outer loose: Precision: 36.28%; Recall: 9.51%; FB1: 15.07
Inner loose: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PERderiv
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
==> PERpart
==============
Outer strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Inner strict: Precision: 0.00%; Recall: 0.00%; FB1: 0.00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment