Where A is a class (e.g. definite article), and B is another class (e.g. indefinite article). O is the null class.
The cat had a dog .
A O O B O O
v1
| Real | Predicted | Verdict |
|---|---|---|
| A | A | TP for A |
| A | B | FP for A |
| A | O | FN for A |
| B | A | FP for B |
| B | B | TP for B |
| B | O | FN for B |
| O | A | FP for O |
| O | B | FP for O |
| O | O | IGNORE |
v2
| Real | Predicted | Verdict |
|---|---|---|
| A | A | TP for A |
| A | B | FP for A |
| A | O | FN for A |
| B | A | FP for B |
| B | B | TP for B |
| B | O | FN for B |
| O | A | ERRANT for A |
| O | B | ERRANT for B |
| O | O | IGNORE |
- TODO: How to include identity vs. correction?l
- TODO: How to include ACCEPT vs. REJECT?
- TODO: What about partially correct spans?