Skip to content

Instantly share code, notes, and snippets.

@cyberandy

cyberandy/wordlift-graphql-data-agent-powered-by-llamaindex-a-simple-demo.ipynb

Last active September 26, 2023 14:05
Show Gist options
  • Save cyberandy/2f000e6037fe7ee174bed5a0bc2a98d0 to your computer and use it in GitHub Desktop.
Save cyberandy/2f000e6037fe7ee174bed5a0bc2a98d0 to your computer and use it in GitHub Desktop.
Download ZIP
wordlift-graphql-data-agent-powered-by-llamaindex-a-simple-demo.ipynb
Display the source blob
Display the rendered blob
Raw
wordlift-graphql-data-agent-powered-by-llamaindex-a-simple-demo.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/cyberandy/2f000e6037fe7ee174bed5a0bc2a98d0/wordlift-graphql-data-agent-powered-by-llamaindex-a-simple-demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "65102716-6a98-4251-9bbc-bd341062e221"
},
"source": [
"## A GraphQL Agent Tool for **WordLift**.\n",
"\n",
"Here we'll connect to our GraphQL end-point, download the full schema and pass it to the llama-hub GraphQL tool first.\n",
"\n",
"We'll build a data agent right after.\n",
"</br>\n",
"<table align=\"left\">\n",
" <td>\n",
" <a href=\"https://wordlift.io\">\n",
" <img width=130px src=\"https://wordlift.io/wp-content/uploads/2018/07/logo-assets-510x287.png\" />\n",
" </a>\n",
" </td>\n",
" <td>\n",
" by\n",
" <a href=\"https://wordlift.io/blog/en/entity/andrea-volpini\">\n",
" Andrea Volpini\n",
" </a>\n",
" MIT License\n",
" <br/>\n",
" <br/>\n",
" <i>Last updated: <b>Sept 16th, 2023</b></i>\n",
" </td>\n",
"</table>\n",
"</br>\n",
"</br>\n"
],
"id": "65102716-6a98-4251-9bbc-bd341062e221"
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "BvyJtaQ6b830"
},
"outputs": [],
"source": [
"%%capture\n",
"!pip install llama-index llama-hub\n",
"!python -m pip install graphql-core\n",
"!pip install unstructured"
],
"id": "BvyJtaQ6b830"
},
{
"cell_type": "markdown",
"metadata": {
"id": "cVtrGBErT5xE"
},
"source": [
"## Get the OpenAI API and the WordLift key"
],
"id": "cVtrGBErT5xE"
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "IvAFa_BrTxzz"
},
"outputs": [],
"source": [
"import os, json\n",
"from typing import Iterator, TextIO\n",
"from datetime import datetime\n",
"import pandas as pd\n",
"\n",
"import io\n",
"from googleapiclient.http import MediaIoBaseDownload\n",
"from google.colab import auth\n",
"from googleapiclient.discovery import build"
],
"id": "IvAFa_BrTxzz"
},
{
"cell_type": "markdown",
"source": [
"Here below I will get the key from both WordLift and OpenAI.\n",
"You can skip this cell and simply set:\n",
"\n",
"\n",
"\n",
"```\n",
"_WL_KEY = \"add-your-wordlift-key-here\"\n",
"_OPENAI_KEY = \"add-your-openai-key-here\"\n",
"```\n",
"\n"
],
"metadata": {
"id": "jPuvFxlvw_ox"
},
"id": "jPuvFxlvw_ox"
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "Hbz37g3VT83L"
},
"outputs": [],
"source": [
"auth.authenticate_user()\n",
"drive_service = build('drive', 'v3', cache_discovery=False)\n",
"\n",
"request = drive_service.files().get_media(fileId='1AyccigFWJ10Jx6eqxf_Pbh19YHDoeO3k')\n",
"downloaded = io.BytesIO()\n",
"downloader = MediaIoBaseDownload(downloaded, request)\n",
"done = False\n",
"while done is False:\n",
" # _ is a placeholder for a progress object that we ignore.\n",
" # (Our file is small, so we skip reporting progress.)\n",
" _, done = downloader.next_chunk()\n",
"\n",
"downloaded.seek(0)\n",
"\n",
"# Read the key from Drive (it's a json file) and store it as enviroment variable\n",
"f = json.loads(downloaded.read())\n",
"\n",
"_WL_KEY = f['wordlift_key']\n",
"_OPENAI_KEY = f['openai_key']\n",
"os.environ['OPENAI_API_KEY'] = _OPENAI_KEY\n"
],
"id": "Hbz37g3VT83L"
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "4c6cb675-121b-4f47-b91d-7fcd6e746fd4"
},
"outputs": [],
"source": [
"# Setup OpenAI Agent\n",
"import openai\n",
"from llama_index.agent import OpenAIAgent"
],
"id": "4c6cb675-121b-4f47-b91d-7fcd6e746fd4"
},
{
"cell_type": "markdown",
"metadata": {
"id": "dbd6a409-472b-4136-912b-2112210c7daf"
},
"source": [
"The Agent *should be* able to form simple GraphQL based on our instructions, and additionally provided some extra parsing and formatting for the data. Nice! We are using here out of the box the `GraphQLToolSpec` available in the Llama Hub."
],
"id": "dbd6a409-472b-4136-912b-2112210c7daf"
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "98e45bed-c0ae-40ba-8504-904e56da502a"
},
"outputs": [],
"source": [
"from llama_hub.tools.graphql.base import GraphQLToolSpec\n",
"# Let's authenticate on the WL KG\n",
"url = 'https://api.wordlift.io/graphql'\n",
"headers = {\n",
" 'accept-language': 'en-US,en;q=0.9',\n",
" 'content-type': 'application/json',\n",
" 'Authorization': 'Key ' + _WL_KEY,\n",
"}\n",
"\n",
"graphql_spec = GraphQLToolSpec(\n",
" url=url,\n",
" headers=headers\n",
")\n",
"agent = OpenAIAgent.from_tools(\n",
" graphql_spec.to_tool_list(),\n",
" verbose=True,\n",
")\n"
],
"id": "98e45bed-c0ae-40ba-8504-904e56da502a"
},
{
"cell_type": "markdown",
"source": [
"Now we can ask a simple question using natural language and see the result."
],
"metadata": {
"id": "CwcTdkUzxo45"
},
"id": "CwcTdkUzxo45"
},
{
"cell_type": "code",
"source": [
"print(agent.chat('Please provide me the iri of 5 persons from the graphql server'))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "7A7_p5GhxnPH",
"outputId": "1b1b1029-405c-4f2a-9888-00afa7070eb8"
},
"id": "7A7_p5GhxnPH",
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"=== Calling Function ===\n",
"Calling function: graphql_request with args: {\n",
" \"query\": \"query Persons { persons { iri } }\",\n",
" \"variables\": {},\n",
" \"operation_name\": \"Persons\"\n",
"}\n",
"Got output: {\n",
" \"data\" : {\n",
" \"persons\" : [ {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/tex_willer\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/seneca\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/elena_pavoncello\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/valentina_ferrero\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/roberto_serra\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/luca_conti\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/britney_muller\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/martin_reichhart\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/maria_silvia_sanna\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/matteo_cassese_2\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/doreid_haddad\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/francesco_morelli\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/stephen_wolfram\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/sergey_brin\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/larry_page\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/adam_lynch\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/larry_kim\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/nichola_stott\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/purna_virji\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/michael_king\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/rand_fishkin\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/raffaele_gaito\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/andrea_volpini\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/jeffrey_smith\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/rainer_edlinger\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/alan_turing\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/umberto_eco\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/kevin_kelly\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/matt_mullenweg\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/aesop\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/boris\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/ben_dickson\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/ted_nelson\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/socrates\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/richard_feynman\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/sarah_mccartney\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/nicola_bonora\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/federica_fiorillo\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/paolo_zanzottera\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/annamaria_anelli\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/leonardo_luccone\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/valentina_falcinelli\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/carlotta_politano\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/stefanie_posavec\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/giorgia_lupi\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/fibonacci\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/maria_popova\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/roger_wolcott_sperry\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/jacques_carelman\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/shiyali_ramamrita_ranganathan\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/donald_norman\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/tim_berners-lee\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/roberto_raviola\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/max_bunker\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/giovan_battista_carpi\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/e-_c-_segar\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/marie_kondo\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/carlos_castaneda\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/jean-paul_sartre\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/cindy_krum\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/bill_slawski\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/fady_ramzy\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/sebastian_hellmann\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/laura_celano\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/ray_kurzweil\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/user/payman\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/user/mariajones\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/gennaro_cuofano\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/claudio_salatino\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/jason_barnard\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/sam_isma\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/cameron_conaway\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/richard_wallis\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/amit-singhal\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/kim-renberg-24070\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/silvia-fratini-24250\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/iryna-ierokh-24279\"\n",
" }, {\n",
" \"iri\" : \"http://data.wordlift.io/wl0216/entity/b-23977\"\n",
" } ]\n",
" }\n",
"}\n",
"========================\n",
"Here are the IRIs of 5 persons from the GraphQL server:\n",
"\n",
"1. http://data.wordlift.io/wl0216/entity/tex_willer\n",
"2. http://data.wordlift.io/wl0216/entity/seneca\n",
"3. http://data.wordlift.io/wl0216/entity/elena_pavoncello\n",
"4. http://data.wordlift.io/wl0216/entity/valentina_ferrero\n",
"5. http://data.wordlift.io/wl0216/entity/roberto_serra\n"
]
}
]
}
],
"metadata": {
"colab": {
"provenance": [],
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment