Skip to content

Instantly share code, notes, and snippets.

@Maltimore
Last active August 17, 2016 17:37
Show Gist options
  • Save Maltimore/38cddfab975a783a81e9318696730829 to your computer and use it in GitHub Desktop.
Save Maltimore/38cddfab975a783a81e9318696730829 to your computer and use it in GitHub Desktop.
# coding=utf-8
import numpy as np
import scipy.sparse as sparse
# Evaluation code courtesy of Juan Nunez-Iglesias, taken from
# https://github.com/janelia-flyem/gala/blob/master/gala/evaluate.py
def adapted_rand(seg, gt, all_stats=False):
"""Compute Adapted Rand error as defined by the SNEMI3D contest [1]
Formula is given as 1 - the maximal F-score of the Rand index
(excluding the zero component of the original labels). Adapted
from the SNEMI3D MATLAB script, hence the strange style.
Parameters
----------
seg : np.ndarray
the segmentation to score, where each value is the label at that point
gt : np.ndarray, same shape as seg
the groundtruth to score against, where each value is a label
all_stats : boolean, optional
whether to also return precision and recall as a 3-tuple with rand_error
Returns
-------
are : float
The adapted Rand error; equal to $1 - \frac{2pr}{p + r}$,
where $p$ and $r$ are the precision and recall described below.
prec : float, optional
The adapted Rand precision. (Only returned when `all_stats` is ``True``.)
rec : float, optional
The adapted Rand recall. (Only returned when `all_stats` is ``True``.)
References
----------
[1]: http://brainiac2.mit.edu/SNEMI3D/evaluation
"""
# segA is truth, segB is query
segA = np.ravel(gt)
segB = np.ravel(seg)
# mask to foreground in A
mask = (segA > 0)
segA = segA[mask]
segB = segB[mask]
n = segA.size # number of nonzero pixels in original segA
n_labels_A = np.amax(segA) + 1
n_labels_B = np.amax(segB) + 1
ones_data = np.ones(n)
p_ij = sparse.csr_matrix((ones_data, (segA.ravel(), segB.ravel())),
shape=(n_labels_A, n_labels_B))
# In the paper where adapted rand is proposed, they treat each background
# pixel in segB as a different value (i.e., unique label for each pixel).
# To do this, we sum them differently than others
B_nonzero = p_ij[:, 1:]
B_zero = p_ij[:, 0]
# this is a count
num_zero = B_zero.sum()
# sum of the joint distribution
# separate sum of B>0 and B=0 parts
sum_p_ij = B_nonzero.astype(np.float32).power(2).sum() + num_zero
# these are marginal probabilities
a_i = p_ij.sum(1).astype(np.float32)
b_i = B_nonzero.sum(0).astype(np.float32)
sum_a = np.power(a_i, 2).sum()
sum_b = np.power(b_i, 2).sum() + float(num_zero)
precision = sum_p_ij / sum_b
recall = sum_p_ij / sum_a
fScore = 2.0 * precision * recall / (precision + recall)
are = 1.0 - fScore
if all_stats:
return (are, precision, recall)
else:
return are
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment