Skip to content

Instantly share code, notes, and snippets.

@AFAgarap
Last active September 29, 2019 05:21
Show Gist options
  • Save AFAgarap/ce21ca3060ae12a48dead570610675f1 to your computer and use it in GitHub Desktop.
Save AFAgarap/ce21ca3060ae12a48dead570610675f1 to your computer and use it in GitHub Desktop.
Calculate trust scores : ratio of distance to closest class other than the predicted class to distance to predicted class. Link to blog: https://towardsdatascience.com/how-can-i-trust-you-fb433a06256c?source=friends_link&sk=0af208dc53be2a326d2407577184686b
def score(self, X: np.ndarray, Y: np.ndarray, k: int = 2, dist_type: str = 'point') \
-> Tuple[np.ndarray, np.ndarray]:
d = np.tile(None, (X.shape[0], self.classes)) # init distance matrix: [nb instances, nb classes]
for c in range(self.classes):
d_tmp = self.kdtrees[c].query(X, k=k)[0] # get k nearest neighbors for each class
if dist_type == 'point':
d[:, c] = d_tmp[:, -1]
elif dist_type == 'mean':
d[:, c] = np.mean(d_tmp, axis=1)
sorted_d = np.sort(d, axis=1) # sort distance each instance in batch over classes
# get distance to predicted and closest other class and calculate trust score
d_to_pred = d[range(d.shape[0]), Y]
d_to_closest_not_pred = np.where(sorted_d[:, 0] != d_to_pred, sorted_d[:, 0], sorted_d[:, 1])
trust_score = d_to_closest_not_pred / (d_to_pred + self.eps)
# closest not predicted class
class_closest_not_pred = np.where(d == d_to_closest_not_pred.reshape(-1, 1))[1]
return trust_score, class_closest_not_pred
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment