Annotator.js works on Drupal as two modules and a library (installed, as usual, under /sites/all):
/modules/annotator
/modules/annotation
/libraries/annotator
The files in libraries/annotator are the latest from the Annotator.js repo (https://github.com/openannotation/annotator/releases). Although the modules Infinite Ulysses uses have been significantly changed from the modules you could download on Drupal.org, the Drupal.org Annotator (https://www.drupal.org/project/annotator) and Annotation (https://www.drupal.org/project/annotation) modules pages might still have useful info if you're troubleshooting.