troubleshooting some API requests returning empty sets

testing-annotations.py

import requests
import json

KEY = "YOUR HYPOTHES.IS API KEY"

# fails - no results
URL = "https://www.scienceintheclassroom.org/research-papers/prioritizing-invertebrate-genomics"

# fails - no results
# URL = "https://www.scienceintheclassroom.org/research-papers/whats-major-source-urban-air-pollution"

# works
# URL = "https://www.scienceintheclassroom.org/research-papers/good-science-reproducible-science"

headers = {
    "Host": "hypothes.is",
    "Accept": "application/json",
    "Authorization": "Bearer %s" %(KEY)
}

base_url = "https://hypothes.is/api/search"
search_url = "".join([base_url, "?uri=", URL])

r = requests.get(search_url, headers=headers)
# print(r.headers)
text = r.json()
json = json.dumps(text, ensure_ascii=False)

if text['total'] > 0:
    print(json)
    print("%s rows" % text['total'])
else:
    print("no rows")

Hi Chris,

As per https://twitter.com/judell/status/1080963225909219328, the queries in your Science in the Classroom pages all succeed because they are using DOIs. Which is a great strategy, btw!

The corresponding queries in your gist fail because there's been a metadata publishing hiccup w/respect to the URL/DOI bindings.

In our database we have a table that maps DOIs to URLs. Here is the mapping for your successful example, doi:10.1126/science.aac4716

http://www.scienceintheclassroom.org/research-papers/good-science-reproducible-science

Here are the mappings for doi:10.1126/science.aaq0524 (https://www.scienceintheclassroom.org/research-papers/whats-major-source-urban-air-pollution)

http://science.sciencemag.org/content/359/6377/760
http://www.scienceintheclassroom.org/research-papers/volatile-chemical-products-emerging-largest-petrochemical-source-urban-organic
http://www.scienceintheclassroom.org/research-papers/whats-major-source-air-pollution-cities

And for doi:10.1093/jhered/est084 (https://www.scienceintheclassroom.org/research-papers/prioritizing-invertebrate-genomics)

http://www.scienceintheclassroom.org/research-papers/giga-community-resources

I have a hunch about what's happened here, but let me first ask: Do these URLs ring any bells for you?

Oh, wait, I just realized you're not the publisher! So that's really a question for the Science in the Classroom folks.

FYI, here's another oddity from the same site:

Running the API on the URL https://www.scienceintheclassroom.org/research-papers/how-cocaine-addiction-can-change-brain returns 5 results, but running it on the doi for that page doi:10.1126/science.aaf3700 returns the 83 that it should (or at least that matches what I see when viewing the page and using the standard browser).