Skip to content

Instantly share code, notes, and snippets.

@jermnelson

jermnelson/api-any-linked-data.ipynb

Last active July 19, 2021 18:05
Show Gist options
  • Save jermnelson/334939e5dc344f5b3f3879df24095e6d to your computer and use it in GitHub Desktop.
Save jermnelson/334939e5dc344f5b3f3879df24095e6d to your computer and use it in GitHub Desktop.
Download ZIP
How to use Sinopia to create ANY kind of linked data
Display the source blob
Display the rendered blob
Raw
api-any-linked-data.ipynb
{
"cells": [
{
"cell_type": "markdown",
"id": "ad510017-ff67-4774-8435-c9bc189ab0c1",
"metadata": {},
"source": [
"# How to use Sinopia to create ANY kind of linked data"
]
},
{
"cell_type": "markdown",
"id": "3f6f4fcc-ff84-4f54-9fcf-f91171548063",
"metadata": {},
"source": [
"# What is the Sinopia API?\n",
"The Sinopia's [application programming interface](https://en.wikipedia.org/wiki/API)(API) is an open-source web service for the [Sinopia Editor](https://sinopia.io/) that provides read, create, and update operations for resources managed in the editor. \n",
"\n",
"There are three Sinopia API instances, one for each environment (development, stage, and production) with identical code-bases located in the source code repository on Github at https://github.com/ld4p/sinopia_api. The API environments are hosted on [Amazon Web Services](https://aws.amazon.com/)(AWS) cloud and connect to other custom and AWS services. \n",
"\n",
"The API design uses the [Representational state transfer](https://en.wikipedia.org/wiki/Representational_state_transfer)(REST) pattern for defining the mechanics of using web-based requests (GET, POST, PUT) on Sinopia resources.\n",
"\n",
"> You can run this notebook with MyBinder by clicking on this button\n",
"> [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gist/jermnelson/334939e5dc344f5b3f3879df24095e6d/HEAD)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "40e7ef41-6a40-42a8-bbcb-84d4ef214711",
"metadata": {},
"outputs": [],
"source": [
"dev_api_url = 'https://api.development.sinopia.io'\n",
"stage_api_url = 'https://api.stage.sinopia.io'\n",
"production_api_url = 'https://api.sinopia.io'"
]
},
{
"cell_type": "markdown",
"id": "e9843867-a2ee-456c-a248-3ae94b54507d",
"metadata": {},
"source": [
"### OpenAPI specification\n",
"For avalable functionality and details on how to construct the API calls to the Sinopia API are defined using the [OpenAPI](https://www.openapis.org/) specification and is available at https://ld4p.github.io/sinopia_api/. \n",
"\n",
"When using the Sinopia API, OpenAPI documentation provides critical information on how to construct the JSON-based web request and also provides examples of what is returned for both successful and failed api requests. We will refer back to the OpenAPI through-out the remainder of this tutorial."
]
},
{
"cell_type": "markdown",
"id": "9a48afa0-fc54-4881-a2ba-b82b9766a144",
"metadata": {},
"source": [
"### Interactions with MongoDB and Elasticsearch\n",
"The Sinopia API primary focus is interacting with [DocumentDB][DOCDB] the AWS's[MongoDB](https://www.mongodb.com/) hosted service. The JSON metadata and RDF for Sinopia Resources are stored as MongoDB [Documents](https://docs.mongodb.com/manual/core/document/). When the resource document is added or changed, [DocumentDB][DOCDB] triggers a message in a running AWS task that indexes the document into AWS's [Elasticsearch](https://aws.amazon.com/elasticsearch-service/) to support search in the Editor.\n",
"\n",
"[DOCDB]: https://aws.amazon.com/documentdb/"
]
},
{
"cell_type": "markdown",
"id": "5ecb9d25-426e-404c-a1c6-1f84969124a9",
"metadata": {},
"source": [
"# Reading resources and templates"
]
},
{
"cell_type": "markdown",
"id": "500fd848-2c89-4574-96c2-4660e034799e",
"metadata": {
"tags": []
},
"source": [
"### MongoDB document structure\n",
"A Sinopia resource in MongoDB can be accessed through your web browser, using a command-line program like `curl`, or programmatically in Python using a library like [requests](https://docs.python-requests.org/en/master/). \n",
"\n",
"The resulting JSON that is returned back from Sinopia API in any of these methods is very similar to the actual document that is stored in MongoDB. In the document, the RDF is serialized as JSON-LD and is available in the `data` key.\n",
"\n",
"For this resource https://api.stage.sinopia.io/resource/70063a5d-578e-408b-917d-989611faa1ff in Sinopia stage, here is the a screen-shot when accessing it through the web browser.\n",
"\n",
"![Kernite API Screenshot](kernite-screen-shot.png)\n",
"\n",
"Using the magic command \"!\" to run POSIX command in this Jupyter notebook, we can use `curl` to retrieve the JSON from the API url. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "138d77d9-79cf-4235-9aef-f313d340adb9",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"! curl https://api.stage.sinopia.io/resource/70063a5d-578e-408b-917d-989611faa1ff"
]
},
{
"cell_type": "markdown",
"id": "8b93148c-0fef-4ad6-b641-1bba1f793a21",
"metadata": {},
"source": [
"Finally, we will import the Python's `requests` library and then retrieve this resource using the `get` method:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "108c9a59-fb17-4673-8b71-b56496052ab1",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"import requests\n",
"\n",
"kernite_result = requests.get('https://api.stage.sinopia.io/resource/70063a5d-578e-408b-917d-989611faa1ff')"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7cbc3e32-0f20-42cd-9daa-8e948b4032da",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(kernite_result.status_code)\n",
"print(json.dumps(kernite_result.json(), sort_keys=True, indent=2))"
]
},
{
"cell_type": "markdown",
"id": "5d293204-55f2-45bf-a70a-89b81560aa81",
"metadata": {},
"source": [
"### Document metadata\n",
"Each Sinopia resource stored in MongoDB also contains a number of metadata fields that are also available for use. We can see the properties or keys of the resource from the previous result:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a278e376-cb0a-4ee6-8abc-09a94d064e4d",
"metadata": {},
"outputs": [],
"source": [
"for key in kernite_result.json().keys():\n",
" print(f\"{key}\")"
]
},
{
"cell_type": "markdown",
"id": "ac8cf31e-526b-4d7a-aab2-4024a365a36b",
"metadata": {},
"source": [
"We can retieve any of these values from the `kernite_result.json()` response object. For example we can get the *id*, *uri*, *timestamp*, and *group* like this: "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "89d943ff-6a2c-4346-b107-7ea0557c769e",
"metadata": {},
"outputs": [],
"source": [
"print(f\"ID is {kernite_result.json()['id']}\")\n",
"print(f\"URI is {kernite_result.json()['uri']}\")\n",
"print(f\"Timestamp is {kernite_result.json()['timestamp']}\")\n",
"print(f\"Group is {kernite_result.json()['group']}\")"
]
},
{
"cell_type": "markdown",
"id": "eb3eda7b-e623-430a-9d2b-f92703a0b6b4",
"metadata": {},
"source": [
"### Accessing RDF\n",
"We will use the Python `rdflib` library to construct a RDF graph and then we will parse the JSON-LD of this resource into the new graph. From there we can serialize it into a different format, run SPARQL queries, or other activity or analysis.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9dbfcf5d-0ff3-4da5-b8b8-01942ecae739",
"metadata": {},
"outputs": [],
"source": [
"import rdflib\n",
"\n",
"kernite_graph = rdflib.Graph()\n",
"kernite_graph.parse(data=json.dumps(kernite_result.json()['data']), \n",
" format='json-ld')\n",
"print(f\"Total number of triples: {len(kernite_graph)}\")"
]
},
{
"cell_type": "markdown",
"id": "99e192c2-9656-4177-a65a-7823410bb110",
"metadata": {},
"source": [
"Serialize the graph as Turtle:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3598b417-e811-4767-9a28-7c5b8b7b358e",
"metadata": {},
"outputs": [],
"source": [
"print(kernite_graph.serialize(format='turtle').decode())"
]
},
{
"cell_type": "markdown",
"id": "53cbbc41-eb92-43e0-bce2-1ea83fb8f029",
"metadata": {},
"source": [
"Run a simple SPARQL query on our graph:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "29ed1a4a-2efa-4143-ab22-1c473b516d39",
"metadata": {},
"outputs": [],
"source": [
"for row in kernite_graph.query(\"\"\"PREFIX dcterm: <http://purl.org/dc/terms/> \n",
"SELECT ?contributor ?title\n",
"WHERE {\n",
" ?subject dcterm:contributor ?contributor .\n",
" ?subject dcterm:title ?title .\n",
"}\"\"\"):\n",
" print(f\"Author is {row['contributor']}, title is {row['title']}\")"
]
},
{
"cell_type": "markdown",
"id": "0fe77433-64f2-40d3-aa18-091f55ec2a0a",
"metadata": {},
"source": [
"### Batch reading based on Group or Resource Class\n",
"The Sinopia API allows us to retieve multiple resources at a time using different filters like `group` or `class`. The full list of filters are listed in the OpenAPI specification at https://ld4p.github.io/sinopia_api/#tag/resources/paths/~1resource/get. The API has additional query parameters for limiting the number of resources called `limit` and a `start` parameter that together allows us to retrieve discrete sets of results of a large number of results.\n",
"\n",
"We can retrieve all of the resources in the Music Library Association group through the following GET call:\n",
"\n",
"`https://api.stage.sinopia.io/resource?group=mla`"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b97287c-7e41-44ae-b3d6-ca8d42d615c5",
"metadata": {},
"outputs": [],
"source": [
"mla_result = requests.get('https://api.stage.sinopia.io/resource?group=mla')"
]
},
{
"cell_type": "markdown",
"id": "9cdc572a-1eb5-4014-9eb5-b8295ea2da13",
"metadata": {},
"source": [
"This result has two properties `data` and `links`. The `data` property contains a list of resources while the `links` property is used for retrieving the next result set (if available)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2621cdf0-1ba1-40f0-a939-8f2870ce0efc",
"metadata": {},
"outputs": [],
"source": [
"print(f\"Number of resources {len(mla_result.json().get('data'))}\")\n",
"print(mla_result.json().get('links'))"
]
},
{
"cell_type": "markdown",
"id": "835aec69-2420-48bb-8f87-bc7f5192992a",
"metadata": {},
"source": [
"The `mla` group has more than 25 results that can be retrieve by doing a second GET query using the `next` URL. We can display the any of the first 25 resources by using Python list index notation in the `data` property list:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4ef09cba-6628-40f5-873c-21082d1145d5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"print(json.dumps(mla_result.json().get('data')[15], sort_keys=True, indent=2))"
]
},
{
"cell_type": "markdown",
"id": "fb76c7e2-126c-47df-9099-f83b7458d356",
"metadata": {},
"source": [
"# Security with JWT\n",
"While all resources in Sinopia are currently world accessable (meaning that anyone can access them through the API as we've seen), to create a new RDF resource or edit an existing resource we need a [JSON Web Token](https://jwt.io/) (JWT) that is generated when a user logs into Sinopia through the AWS [Cognito](https://aws.amazon.com/cognito/) service."
]
},
{
"cell_type": "markdown",
"id": "87b9aae1-bcef-4c21-bff6-ff4889444160",
"metadata": {},
"source": [
"### Use API for generating JWT\n",
"To generate the JWT from the command-line is possible through the Sinopia API but we require additional parameters to pass into the API. We need to provide these AWS variables to our command-line API program `bin/authenticate`\n",
"\n",
"- **AWS_PROFILE**\n",
"- **COGNITO_USER_POOL**\n",
"- **COGNITO_CLIENT**\n",
"- **AWS_COGNITO_DOMAIN**\n",
"\n",
"Running this program prompts you for your Sinopia username and then password. \n",
"\n",
"![Authenticate through the command-line](sinopia-api-auth.png)\n",
"\n",
"This generates a JWT called `.cognitoToken` that can then be added to the HTTP headers for **POST** and **PUT** commands. Unfortunately, a couple of the AWS variables should remain secrets but we get this token from our web browser after logging into a specific Sinopia environment. "
]
},
{
"cell_type": "markdown",
"id": "494791ac-22b9-437e-b044-ca0b1b9b2aca",
"metadata": {},
"source": [
"### Browser-based JWT\n",
"We can copy a valid JWT from variables contained in our web-browser that we can then use. We will demostrate this method using Firefox but you should be able to take a similar approach with Chrome. \n",
"\n",
"#### 1. Log into Sinopia Stage (or other environment)\n",
"![Login to Sinopia Stage](sinopia-login-screen.png)\n",
"\n",
"#### 2. Open the Developer Tools \n",
"![Open Firefox Developer](sinopia-firefox-open-dev-tools.png)\n",
"\n",
"#### 3. Click on the Application tab in Developer Tools\n",
"After clicking on the **Application** tab, on the left side, under the **Local Storage** look and click on the `https://stage.sinopia.io/` and in the main plain select the key that starts with `CognitoIdentityService...` and copy the value. This is your JWT.\n",
"\n",
"![Application Local Storage](sinopia-stage-jwt-in-dev-tools.png)"
]
},
{
"cell_type": "markdown",
"id": "33c1c8ad-8bb8-4049-a603-4380ee1e24e4",
"metadata": {},
"source": [
"# Creating Resources in RDF the Editor and in a Jupyter notebook\n",
"If you have external RDF that you like to add to Sinopia, one option is to use the `Load RDF` tab in the editor. Here we will take a small subset of triples from Wikidata and load it into Sinopia stage."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "211fa0b2-c1eb-42d1-9f43-a1d1f718546c",
"metadata": {},
"outputs": [],
"source": [
"example_wikidata_rdf = \"\"\"@prefix wd: <http://www.wikidata.org/entity/> .\n",
"@prefix schema: <http://schema.org/> .\n",
"\n",
"<https://en.wikipedia.org/wiki/Parable_of_the_Sower_(novel)> a schema:Article ;\n",
"\tschema:about wd:Q3823447 ;\n",
"\tschema:inLanguage \"en\" ;\n",
"\tschema:isPartOf <https://en.wikipedia.org/> ;\n",
"\tschema:name \"Parable of the Sower (novel)\"@en .\"\"\""
]
},
{
"cell_type": "markdown",
"id": "585957ba-39af-44c0-96f6-0cb871b42eb3",
"metadata": {},
"source": [
"### Load RDF through the Editor\n",
"\n",
"#### 1. Click on Load RDF tab and Paste RDF into Text Area and paste Base URI\n",
"![Sinopa Load RDF Tab](sinopia-load-rdf-tab.png)\n",
"\n",
"#### 2. Save and Select Resource Template\n",
"![Sinopia Load RDF Template Selection](sinopia-load-rdf-template-selection.png)\n",
"\n",
"#### 3. See Loaded RDF in Sinopia Editor\n",
"![Sinopa Loaded RDF in Editor](sinopia-load-rdf-editor.png)"
]
},
{
"cell_type": "markdown",
"id": "e11c9928-2f2a-471d-97ae-79f2f7538a0d",
"metadata": {},
"source": [
"### Structure of HTTP POST Request\n",
"We can also use the JWT from the previous step to construct a Sinopia API HTTP POST request in Python. We first need to construct an authentication header, serialize our RDF, and add some document metadata to create a new RDF resource in Sinopia."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3cb5e7ba-a56c-433d-9eac-c03b6770a29b",
"metadata": {},
"outputs": [],
"source": [
"stage_jwt = 'eyJraWQiOiJ5SnczaUFaOWsrRVdQekxhakVFbVwvM1FzdUZzT0hcL2NucWdyWWhcL2VsRHZvPSIsImFsZyI6IlJTMjU2In0.eyJzdWIiOiIxYzJiNDFkNS1lYzE5LTQxY2ItOWNkOC03OTA3NDFkNDhmYzEiLCJldmVudF9pZCI6IjZlNTMwOTVhLTUyOGMtNDVlMC1iZGY3LWE4YTE2ODEyNzA3MyIsInRva2VuX3VzZSI6ImFjY2VzcyIsInNjb3BlIjoiYXdzLmNvZ25pdG8uc2lnbmluLnVzZXIuYWRtaW4iLCJhdXRoX3RpbWUiOjE2MjY0ODE4OTksImlzcyI6Imh0dHBzOlwvXC9jb2duaXRvLWlkcC51cy13ZXN0LTIuYW1hem9uYXdzLmNvbVwvdXMtd2VzdC0yX2lsTVFXME0wUiIsImV4cCI6MTYyNjQ4NTQ5OSwiaWF0IjoxNjI2NDgxODk5LCJqdGkiOiJjYTViNjUxZi03NjZiLTQwNDYtYTQ2ZC0wNTY2MWI0NmE0OTIiLCJjbGllbnRfaWQiOiIyM2wzMHBidTc0cHVzZ2g3Y210dWZiOGQwNSIsInVzZXJuYW1lIjoianBuZWxzb24ifQ.aqGd-nJLEU3_g6a5b6fLUstJpa7vFJFPsi0WG2TGeraim8QRfpR-FrqbiyyR1-lS9L0Db9eSm3ZXuAquwngQ74pNCnuyzl-cKxjztHXyTSS5SDwdXvXCOrjOQCgZXDwSp_44RPYnX2mYIT-7adpA5F3tsbrPZqEGIBJ_eA5s9pmtoDUzuX2P8kOt9j_XoDVbgJaLdTvlSrAvHmDUtfa8xaUvX2nPVhXeqw-1H6zO6yqXtV7WVZkEzYCQdLCHg6h1x6KE61RtQf-QDlgoN6_NY6PQFqPVvdXrTfet1uE78rFP1s7mcNcWXL0ET9R5xB4HQIbI6GIW-uOcTKrCE2Z-8g'\n",
"headers = {\n",
" \"Authorization\": f\"Bearer {stage_jwt}\",\n",
" \"Content-Type\": \"application/json\"\n",
"}"
]
},
{
"cell_type": "markdown",
"id": "b2ea83ee-a25e-4fe4-a3c7-a1f909e70248",
"metadata": {},
"source": [
"We will use a portion of the [RDF wikidata entry](https://www.wikidata.org/wiki/Special:EntityData/Q25217632.ttl) for *The Fifth Season* the first novel of N.K. Jemisin's **The Broken Earth** trilogy, to construct our JSON-LD along with the additional document metadata. We will make some changes to the RDF to better align the schema.org triples with the existing resource template `sinopia:resourceTemplate:schema:Book` in Sinopia stage."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "17c65625-fcf2-4b8f-b5c0-f68410275b34",
"metadata": {},
"outputs": [],
"source": [
"import uuid\n",
"random_uuid = uuid.uuid4()\n",
"sinopia_stage_url = f\"https://api.stage.sinopia.io/resource/{random_uuid}\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7044ba47-3e82-4b5c-a4ba-df00a314ac59",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_ttl = f\"\"\"@prefix skos: <http://www.w3.org/2004/02/skos/core#> .\n",
"@prefix wd: <http://www.wikidata.org/entity/> .\n",
"@prefix schema: <http://schema.org/> .\n",
"@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
"\n",
"<https://en.wikipedia.org/wiki/The_Fifth_Season_(novel)> a schema:Article ;\n",
" schema:about <{sinopia_stage_url}> ;\n",
" schema:inLanguage \"en\" ;\n",
" schema:isPartOf <https://en.wikipedia.org/> ;\n",
" schema:name \"The Fifth Season (novel)\"@en .\n",
" \n",
"<{sinopia_stage_url}> a schema:Book ;\n",
" schema:sameAs wd:Q25217632 ;\n",
" rdfs:label \"The Fifth Season\"@en ;\n",
" skos:prefLabel \"The Fifth Season\"@en ;\n",
" schema:name \"The Fifth Season\"@en ;\n",
" rdfs:label \"La quinta estación\"@es ;\n",
" skos:prefLabel \"La quinta estación\"@es ;\n",
" schema:name \"La quinta estación\"@es ;\n",
" schema:description \"2015 novel by N. K. Jemisin\"@en,\n",
"\t\t\"novela de N. K. Jemisin\"@es ;\n",
" schema:author wd:Q2427544 .\n",
" \n",
"wd:Q2427544 a schema:Person ;\n",
" rdfs:label \"N. K. Jemisin\"@en ;\n",
" skos:prefLabel \"N. K. Jemisin\"@en ;\n",
" schema:name \"N. K. Jemisin\"@en .\n",
"\"\"\""
]
},
{
"cell_type": "markdown",
"id": "6600771e-fa62-4c82-bbdb-edacb2d3c3fb",
"metadata": {},
"source": [
"We first create an RDF graph, register a couple of namespaces in the graph, and then parse the turtle RDF. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f1dc059e-932b-42cc-b736-a1e02653fcb4",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_graph = rdflib.Graph()\n",
"fifth_season_graph.namespace_manager.bind(\"wd\", \"http://www.wikidata.org/entity/\")\n",
"fifth_season_graph.namespace_manager.bind(\"rdfs\", \"http://www.w3.org/2000/01/rdf-schema#\")\n",
"fifth_season_graph.namespace_manager.bind(\"schema\", \"http://schema.org/\")\n",
"fifth_season_graph.namespace_manager.bind(\"skos\", \"http://www.w3.org/2004/02/skos/core#\")\n",
"fifth_season_graph.parse(data=fifth_season_ttl, format='turtle')"
]
},
{
"cell_type": "markdown",
"id": "01714599-3bd9-4d16-95f3-e5778158ee21",
"metadata": {},
"source": [
"### POST new resource to Sinopia API\n",
"We then construct the body of our HTTP POST request and send it to the stage Sinopia API along with a random UUID in the URL. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3c6fd151-13ff-4347-8cb6-03cb446dfa28",
"metadata": {},
"outputs": [],
"source": [
"body = { \"data\": fifth_season_graph.serialize(format='json-ld').decode(),\n",
" \"user\": \"jpnelson\",\n",
" \"group\": \"other\" }"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cda1d677-8237-4880-a967-ea2c317e2ad4",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_result = requests.post(sinopia_stage_url,\n",
" data=json.dumps(body),\n",
" headers=headers)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f0e6c9cb-4e05-434b-91c3-34269478cb54",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_result.status_code"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d323cc00-6550-4688-84ac-39ff73f976dc",
"metadata": {},
"outputs": [],
"source": [
"print(json.dumps(fifth_season_result.json(), indent=2))"
]
},
{
"cell_type": "markdown",
"id": "6a332d89-af4e-4b3c-afbd-5602691a2488",
"metadata": {},
"source": [
"### Batch loading of Resources\n",
"To load a batch of different resources using the Sinopia API is similar to adding a single resource. We will create two other RDF resources for the remaining two books in **Broken Earth Trilogy** and post both of them to stage using the Sinopia API."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e659a4d-7925-4275-87d2-3299bf12bc1f",
"metadata": {},
"outputs": [],
"source": [
"# Second book in the trilogy\n",
"obelisk_gate_uri = f\"https://api.stage.sinopia.io/resource/{uuid.uuid4()}\"\n",
"obelisk_gate_ttl = f\"\"\"@prefix skos: <http://www.w3.org/2004/02/skos/core#> .\n",
"@prefix wd: <http://www.wikidata.org/entity/> .\n",
"@prefix schema: <http://schema.org/> .\n",
"@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
"\n",
"<https://en.wikipedia.org/wiki/The_Obelisk_Gate> a schema:Article ;\n",
" schema:about <{obelisk_gate_uri}> ;\n",
" schema:inLanguage \"en\" ;\n",
" schema:isPartOf <https://en.wikipedia.org/> ;\n",
" schema:name \"The Obelisk Gate\"@en .\n",
"\n",
"<{obelisk_gate_uri}> a schema:Book ;\n",
" schema:sameAs wd:Q30337352 ;\n",
" rdfs:label \"The Obelisk Gate\"@en ;\n",
" skos:prefLabel \"The Obelisk Gate\"@en ;\n",
" schema:name \"The Obelisk Gate\"@en ;\n",
" schema:description \"2016 novel by N. K. Jemisin\"@en,\n",
" \"novela de Nora K. Jemisin\"@es ;\n",
" schema:author wd:Q2427544 .\n",
" \n",
"wd:Q2427544 a schema:Person ;\n",
" rdfs:label \"N. K. Jemisin\"@en ;\n",
" skos:prefLabel \"N. K. Jemisin\"@en ;\n",
" schema:name \"N. K. Jemisin\"@en .\n",
"\"\"\"\n",
"\n",
"stone_sky_uri = f\"https://api.stage.sinopia.io/resource/{uuid.uuid4()}\"\n",
"stone_sky_ttl = f\"\"\"@prefix skos: <http://www.w3.org/2004/02/skos/core#> .\n",
"@prefix wd: <http://www.wikidata.org/entity/> .\n",
"@prefix schema: <http://schema.org/> .\n",
"@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
"\n",
"<https://en.wikipedia.org/wiki/The_Stone_Sky> a schema:Article ;\n",
" schema:about <{stone_sky_uri}> ;\n",
" schema:inLanguage \"en\" ;\n",
" schema:isPartOf <https://en.wikipedia.org/> ;\n",
" schema:name \"The Stone Sky\"@en .\n",
" \n",
"<{stone_sky_uri}> a schema:Book ;\n",
" schema:sameAs wd:Q39058554 ;\n",
" rdfs:label \"The Stone Sky\"@en ;\n",
" skos:prefLabel \"The Stone Sky\"@en ;\n",
" schema:name \"The Stone Sky\"@en ;\n",
" rdfs:label \"El cielo de piedra\"@es ;\n",
" skos:prefLabel \"El cielo de piedra\"@es ;\n",
" schema:name \"El cielo de piedra\"@es ;\n",
" schema:description \"2017 novel by N. K. Jemisin\"@en,\n",
" \"novela de N. K. Jemisin\"@es ;\n",
" schema:author wd:Q2427544 .\n",
" \n",
"wd:Q2427544 a schema:Person ;\n",
" rdfs:label \"N. K. Jemisin\"@en ;\n",
" skos:prefLabel \"N. K. Jemisin\"@en ;\n",
" schema:name \"N. K. Jemisin\"@en .\n",
"\"\"\""
]
},
{
"cell_type": "markdown",
"id": "04538d3a-ddee-41f8-ae90-a4fb217bc84f",
"metadata": {},
"source": [
"Finally we will loop through a list made up of the RDF and uris for each of remaining books and post to the Sinopia API"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e2ab3e2f-3eed-439d-9da3-7a956d11aa16",
"metadata": {},
"outputs": [],
"source": [
"for uri, ttl in [(obelisk_gate_uri, obelisk_gate_uri),\n",
" (stone_sky_uri, stone_sky_ttl)]:\n",
" graph = rdflib.Graph()\n",
" graph.namespace_manager.bind(\"wd\", \"http://www.wikidata.org/entity/\")\n",
" graph.namespace_manager.bind(\"rdfs\", \"http://www.w3.org/2000/01/rdf-schema#\")\n",
" graph.namespace_manager.bind(\"schema\", \"http://schema.org/\")\n",
" graph.namespace_manager.bind(\"skos\", \"http://www.w3.org/2004/02/skos/core#\")\n",
" graph.parse(data=ttl, format='turtle')\n",
" body = {\n",
" \"data\": graph.serialize(format='json-ld').decode(),\n",
" \"user\": \"jpnelson\",\n",
" \"group\": \"other\"\n",
" }\n",
" post_result = requests.post(uri,\n",
" data=json.dumps(body),\n",
" headers=headers)\n",
" print(f\"Result of post {post_result.status_code}\")"
]
},
{
"cell_type": "markdown",
"id": "818f4ac5-ee69-4b47-9449-867a30c42a29",
"metadata": {},
"source": [
"# Editing Existing Resources and Templates\n",
"Editing or updating an existing resource follows a similar process only this time instead of using **POST**, we will use **PUT**. For our three books we added, we forgot to include the *templateId* metadata as well as the Sinopia triple for the resource template."
]
},
{
"cell_type": "markdown",
"id": "63bb0661-3fa7-46d8-b098-135a40f20722",
"metadata": {},
"source": [
"### Structure of HTTP PUT Request\n",
"We can use the same JWT and headers that we used for the **POST** request for our **PUT** request."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ec49c1f4-89f3-44e0-bb70-dfa72847e2ce",
"metadata": {},
"outputs": [],
"source": [
"print(headers)"
]
},
{
"cell_type": "markdown",
"id": "5dd32dcc-96c5-4ee7-bd06-0d79ed088c2b",
"metadata": {},
"source": [
"### PUT updated resource to Sinopia API\n",
"We retrieve the resource for https://api.stage.sinopia.io/resource/57c0a287-2306-4e9a-b156-d910773daf30 resource from Stage, add our `templateId` to the metadata, parse the JSON-LD into a new graph and add a triple with http://sinopia.io/vocabulary/hasResourceTemplate as the predicate."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ee9f9eb9-b763-425f-8edf-4e1300a9d135",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_result = requests.get(sinopia_stage_url)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1d5a64e1-6719-4a4e-81bd-1ca16328e063",
"metadata": {},
"outputs": [],
"source": [
"fifth_season_graph = rdflib.Graph()\n",
"existing_body = fifth_season_result.json()\n",
"existing_body['templateId'] = 'sinopia:resourceTemplate:schema:Book'\n",
"fifth_season_graph.parse(existing_body.get('data'),\n",
" format='turtle')\n",
"fifth_season_graph.add((rdflib.URIRef('https://api.stage.sinopia.io/resource/57c0a287-2306-4e9a-b156-d910773daf30'),\n",
" rdflib.URIRef('http://sinopia.io/vocabulary/hasResourceTemplate'),\n",
" rdflib.Literal('sinopia:resourceTemplate:schema:Book'))\n",
"existing_body['data'] = fifth_season_graph.serialize(format='json-ld').decode()\n",
"fifth_season_put_result = requests.put('https://api.stage.sinopia.io/resource/57c0a287-2306-4e9a-b156-d910773daf30',\n",
" data=json.dumps(existing_body),\n",
" headers=headers)\n",
"print(fifth_season_put_result.status_code)"
]
},
{
"cell_type": "markdown",
"id": "a4f17150-5fdf-4b39-a480-30f11bca9631",
"metadata": {},
"source": [
"### Batch editing of Resources\n",
"We will now update the remaining two books in the trilogy with the TemplateID using a loop."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "469eb1c5-df10-41de-bec2-1ff57c0f4850",
"metadata": {},
"outputs": [],
"source": [
"for uri in [obelisk_gate_uri, stone_sky_uri]:\n",
" existing_resource = requests.get(uri).json()\n",
" graph = rdflib.Graph()\n",
" graph.parse(data=existing_resource.get('data'), format='json-ld')\n",
" graph.add((rdflib.URIRef(uri),\n",
" rdflib.URIRef('http://sinopia.io/vocabulary/hasResourceTemplate'),\n",
" rdflib.Literal('sinopia:resourceTemplate:schema:Book')))\n",
" existing_resource['templateId'] = 'sinopia:resourceTemplate:schema:Book'\n",
" existing_resource['data'] = graph.serialize(format='json-ld').decode()\n",
" put_response = requests.put(uri,\n",
" data=json.dumps(existing_resource),\n",
" headers=headers)\n",
" print(put_response.status_code)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment