about_te_errors.markdown

Allow Users to Report Problematic Summon Topic Explorer Entries

Summon's Topic Explorer shows relevant reference material in the right-hand pane to users doing broad searches. The location of this content is getting more useful as Google adds more and more features to it's Knowledge Graph, which shows relevant information (including reference content, store hours and traffic, content from Gmail and calendar and more). In the past, the right-hand side of a page was largely ignored by users, since it was the home of banner ads and other useless bits of content (and it still is in many places). But Google's push to take back the right side pane means our users are more and more likely to see content there, especially when searching. So it's never more important that we get things like Topic Explorer right.

I've been recording Topic Explorer entries that are shown to users since November, and have analyzed over 8,000 search query/Topic Explorer pairs (that's less than half the entries I have in my data set). The overwhelming majority are pretty good. In my analysis, 93% of Topic Explorer entries were on the mark or close. But there are still times when the Topic Explorer entry is off, and I wanted a way for our users to help us know when they've found a problem, especially if they make it seem like the tool is returning biased results.

Today I launched a new feature in GVSU's Summon instance that gives our users the option to anonymously report a mismatch between their search terms and the Topic Explorer results.

The script styles the "Read More" link as a button, and then adds two more links below, one on the left and another on the right. The first link allows users to "Report a problem with this result." Clicking that link will send a request to a PHP script that adds the data to our project management system without recording any identifying information about the user (except in cases where the search terms could identify them, if that ever happens with Summon). The script then changes the text of the link to let them know we got their request and will look into improving it. I've already shared my 8,000 analyzed records (along with flagged innaccuracies and topics that appear biased) with ProQuest, and they're working to improve the Topic Explorer tool. This feature will give us another way to spot problematic searches, and also signal to our users that if there is a problem, we want to make it better.

The second link brings up a small modal window that explains how the entries in the Topic Explorer are chosen, without getting technical or ever using the word "algorithm." This is meant to help contextualize this contextual information.

Finally, I'm finishing up an analysis of the Topic Explorer records to be published soon. I think it's geting more and more important to push for quality not just in design and UIs and collection coverage, but also for accuracy and fairness in the algorithms that drive so much of library software. If you're interested in that project, feel free to drop me a line. I'd love to chat more about it.

If you're interested in the code behind the changes, I've pulled out the relevant bits and put them on Github.

installation.markdown

Installation

The code for this modification only works with Summon 2.0, when you have Topic Explorer entries turned on.

The code for this modification is in two files, one in JavaScript (and jQuery) and the other in PHP. You'll need a web server to host these files.

1. Customize the te_notifier.php file with information about your installation of Summon. First, set the correct URLs for your Summon installation. Replace the SUMMONCODE with the prefix of your Summon site. For instance., our Summon prefix is "gvsu" so our lines look like this:

   header('Access-Control-Allow-Origin: http://gvsu.summon.serialssolutions.com');
header('Access-Control-Allow-Origin: https://gvsu.summon.serialssolutions.com');

   This will allow only requests that originate from your Summon site to use the script.

2. Add the address you want your emails to come from, as well as the emails you want to be notified at. You'll find the FROM@EMAIL token on lines 18 and 19 of te_notifier.php, and the TO@EMAIL token on line 28. (Make sure the token on line 28 is still surrounded by the single quotes.)

3. If you have another way to report problems, you'll want to change the URL in the error message on line 33. (or just remove it if you don't.)

4. Upload te_notifier.php to a web server that can execute PHP files, and note the URL.

5. Now edit the te_report_summon2.js file. The only thing that has to be changed is the path to the PHP file you uploaded in step 4. Change that on line 30. You probably also want to change the Backup URL to send folks to if the AJAX call fails when they want to report a problem.

6. Upload the JavaScript file to your server. Then, in the Summon Admin Console, add the URL of the file to the Custom JavaScript field near the bottom of the settings.

7. Test it out!

If you run into problems, drop me a line at reidsmam@gvsu.edu and I'll do my best to help you out. The more of us that can report problems to ProQuest, the better the algorithm will be!

te_notifier.php

<?php
// Change the SUMMONCODE to your Summon instance prefix
header('Access-Control-Allow-Origin: http://SUMMONCODE.summon.serialssolutions.com'); 
header('Access-Control-Allow-Origin: https://SUMMONCODE.summon.serialssolutions.com');  
// Don't echo error to the user in production.
error_reporting(0);

$m = NULL;
$message = stripslashes($_POST['feedback']);
$url = stripslashes($_POST['url']);

// If there isa refering URL, add it to the message
if(isset($url)) {
        $message = $message . "\n\rProblem URL: " . urldecode($url);
}

// Build the headers
$headers = "From: FROM@EMAIL\r\n";
$headers .= "Reply-To: FROM@EMAIL\r\n";
$headers .= "X-Mailer: PHP/".phpversion();

// Build the message. In our production version we check for some bad strings to prevent abuse.
$error_report = $message;
$error_report .= "\n\n" . 'From: Summon User'; // Don't ask user for personal information.

// Attempt to send the mail, then set the message variable $m to success or error.
if($m == NULL) { // There have been no errors, send the email
  if(mail('TO@EMAIL', 'Topic Explorer Issue', $error_report, $headers)) {
    // Set the link text in Summon - can pass HTML
    echo 'Thank you. We&#8217;re looking into it!'; 
  } else {
    // Request failed.
    echo 'Something went wrong. You can still <a href="BACKUP/PLAN/URL">report the problem manually</a>.';
  }
}

te_report_summon2.js

$(document).ready(function() {
 // Set the Angular.js scope 
setTimeout(function() {
  libMyScope = angular.element('html').scope();
  libInitWithScope( );
}, 1000);

function addTEHelp() {
  // Capture the search query and other variables
  var searchQuery = libGetQueryVariable('q', window.location.href), thisUrl = encodeURIComponent(window.location);

  // Sometimes the search query is in the variable q, sometimes in s.q
  if((typeof searchQuery === 'undefined') || (searchQuery == null)) {
    searchQuery = libGetQueryVariable('s.q', window.location.search);
  }

  // Check to see if there is a Topic Explorer entry
  if($('[aria-label="Topic Summary"]').length > 0) { // Yes

    var topicTitle = $('div.rightBar[aria-label="Topic Summary"]').find('h1:first').text();
    var topicFrom = $('div.rightBar[aria-label="Topic Summary"]').find('div.from.ng-binding').text();
    var topicSummary = $('div.rightBar[aria-label="Topic Summary"]').find('div.snippet.hidden-phone.ng-binding').text();
    console.log('This search has a Topic Explorer entry of ' + topicTitle + ' from ' + topicFrom);

    // Broken up to be readable
    var topicExplorerButton = '<p class="gvsu-te-help" style="margin:.75em 0; border-bottom: 1px solid #999; padding: .75em 0;">';
    topicExplorerButton += '<a href="http://labs.library.gvsu.edu/status/?problem&url=' + thisUrl + '" class="te_problem">';
    topicExplorerButton += 'Report a Problem with this Result</a>';
    topicExplorerButton += '<a style="float:right;display: inline-block; margin-left: 1em;" href="#" id="te-why">Why is this here?</a></p>';
      
    $('div.rightBar[aria-label="Topic Summary"]').find('.sourceLink.customPrimaryLink').addClass('btn').css('margin-top','.75em').addClass('btn-default').parent('div').append(topicExplorerButton);

    $('.te_problem').click(function(e) {
      e.preventDefault();

      var problemMessage = 'Search: ' + searchQuery + "\n\r" + 'Topic: ' + topicTitle + "\n\r" + 'Source: ' + topicFrom +  "\n" + 'Summary: ' + topicSummary;
      var teProblem = $.ajax({
        url: "PATH/TO/PHP",
        method: "POST",
        data: { feedback : problemMessage, url: thisUrl }
      });
       
      teProblem.done(function( msg ) {
        console.log('Saved search query: ' + msg);
        // Change the link text, remove click handler
        $('.te_problem').css('color', 'green').removeClass('te_problem').html(msg);
      });
      teProblem.fail(function( jqXHR, textStatus ) {
        console.log( "Request failed: " + textStatus );
      });
    });
    $('#te-why').click(function(e) {// What is this modal window
      e.preventDefault();
    
      // Add modal window styles
      $('body').append('<style>.overlay p { margin-bottom: .6em; }.modal-box{font-size:1.2em;width:30em;background-color: #fff;padding:1em;position:fixed;top:20%;left:25%;z-index:1000;box-shadow:5px;border:2px solid #bbb;}.close-button{cursor:pointer;    position: absolute;top: -1em;right: 1em;}@media screen and (max-width:700px){.modal-box{width:90%;left:0;}}body {height:100%;position:relative;}.overlay{position:absolute;left:0;top:0;background: rgba(0,0,0,.5);z-index:500;width:100%;height:100%;}.line{margin-top:2em;}.line div { text-align:center;}.close-button{margin-top:1.5em;color:#575757;}</style>');
    
      // Insert a modal dialog box to direct users to Document Delivery
      $("body").append('<div class="overlay"><div class="modal-box"><h4 style="text-align:center;margin-bottom: 1em;">What is this?</h4><p>This library search attempts to match your search terms with reference material related to the topic you are interested in.</p><p>This process is automated but isn&#8217;t always perfect. At times, words in your search or synonyms might cause unrelated topics to appear here.</p><p>If that&#8217;s the case, you can always <a href="http://labs.library.gvsu.edu/status/?problem&url=' + thisUrl + '">let us know there is a problem</a>. We work closely with the team that built this search tool to improve these results.</p><div class="close-button">[x]</div></div></div>');

      $(".close-button").click(function() {
        $(".overlay").hide();
      });
    });
  }
}

function libInitWithScope( ){
    console.log('initialising...');

    // WATCH FOR RESULTS FEED CHANGES...
    libMyScope.$watchCollection('feed', function(){
      // give AngularJS time to update the DOM
      setTimeout(function() { 
        addTEHelp();
      }, 1000);
      console.log('Scope.feed changed! - Refreshed Topic Explorer Help');
    });

}

});