tteofili

Instructions for evaluating ER explanations with CERTEM

1. create a new Google Colab Notebook
2. open the CERTEM-demo Google Colab Notebook at https://colab.research.google.com/drive/1e9L4Edbm3_X4V_xH7EFkxsmT3wE4Qwca?usp=sharing
3. copy each cell from CERTEM-demo.ipynb into your newly created notebook (in the very same order)
4. Run all cells (Runtime -> Run all)
5. Once all cells are loaded you should see something like
6. Select one of the datasets in the dropdown menu, either BA or AB (avoid IA because of visualization issues)
7. Select one of the existing models by checking one of the boxes DeepER, DeepMatcher, Ditto.

tteofili

tteofili

apiVersion: image.openshift.io/v1
kind: ImageStream
metadata:
  labels:
    opendatahub.io/notebook-image: "true"
  annotations:
    opendatahub.io/notebook-image-url: "https://github.com/thoth-station/s2i-generic-data-science-notebook"
    opendatahub.io/notebook-image-name: "Standard Data Science"
    opendatahub.io/notebook-image-desc: "Jupyter notebook data science image with TrustyAI library installed"
  name: s2i-generic-data-science-notebook

tteofili

/*
 * Anserini: A Lucene toolkit for reproducible information retrieval research
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software

tteofili

/*
 * Anserini: A Lucene toolkit for reproducible information retrieval research
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software

tteofili

tteofili

tteofili

/*
 * Copyright 2021 Red Hat, Inc. and/or its affiliates.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *       http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software

tteofili

/*
 * Copyright 2021 Red Hat, Inc. and/or its affiliates.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *       http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software

tteofili