Skip to content

Instantly share code, notes, and snippets.

@sergiocima

sergiocima/chatbot-per-scire.ipynb

Created April 2, 2023 23:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sergiocima/4fc6b1db57026c03e6ac7ef85b86f207 to your computer and use it in GitHub Desktop.
Save sergiocima/4fc6b1db57026c03e6ac7ef85b86f207 to your computer and use it in GitHub Desktop.
Download ZIP
Chatbot per Scire.ipynb
Display the source blob
Display the rendered blob
Raw
chatbot-per-scire.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyO3re10JJAyngipa8qlVttL",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/sergiocima/4fc6b1db57026c03e6ac7ef85b86f207/chatbot-per-scire.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"Seguo le istruzioni di questo articolo https://bootcamp.uxdesign.cc/a-step-by-step-guide-to-building-a-chatbot-based-on-your-own-documents-with-gpt-2d550534eea5\n",
"\n"
],
"metadata": {
"id": "iNFQC3gdNWdJ"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "nlSAbXY9azyk",
"outputId": "063df8b7-8bb6-4be5-e8bc-e1baeb7da59d"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Collecting llama-index\n",
" Downloading llama_index-0.4.29.tar.gz (140 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.7/140.7 KB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting dataclasses_json\n",
" Downloading dataclasses_json-0.5.7-py3-none-any.whl (25 kB)\n",
"Collecting langchain\n",
" Downloading langchain-0.0.113-py3-none-any.whl (396 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m396.0/396.0 KB\u001b[0m \u001b[31m13.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from llama-index) (1.22.4)\n",
"Requirement already satisfied: tenacity<9.0.0,>=8.2.0 in /usr/local/lib/python3.9/dist-packages (from llama-index) (8.2.2)\n",
"Collecting openai>=0.26.4\n",
" Downloading openai-0.27.2-py3-none-any.whl (70 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.1/70.1 KB\u001b[0m \u001b[31m4.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.9/dist-packages (from llama-index) (1.4.4)\n",
"Collecting tiktoken\n",
" Downloading tiktoken-0.3.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m37.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from openai>=0.26.4->llama-index) (4.65.0)\n",
"Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.9/dist-packages (from openai>=0.26.4->llama-index) (2.25.1)\n",
"Collecting aiohttp\n",
" Downloading aiohttp-3.8.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m10.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting marshmallow-enum<2.0.0,>=1.5.1\n",
" Downloading marshmallow_enum-1.5.1-py2.py3-none-any.whl (4.2 kB)\n",
"Collecting typing-inspect>=0.4.0\n",
" Downloading typing_inspect-0.8.0-py3-none-any.whl (8.7 kB)\n",
"Collecting marshmallow<4.0.0,>=3.3.0\n",
" Downloading marshmallow-3.19.0-py3-none-any.whl (49 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.1/49.1 KB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: PyYAML<7,>=6 in /usr/local/lib/python3.9/dist-packages (from langchain->llama-index) (6.0)\n",
"Requirement already satisfied: pydantic<2,>=1 in /usr/local/lib/python3.9/dist-packages (from langchain->llama-index) (1.10.6)\n",
"Requirement already satisfied: SQLAlchemy<2,>=1 in /usr/local/lib/python3.9/dist-packages (from langchain->llama-index) (1.4.46)\n",
"Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.9/dist-packages (from pandas->llama-index) (2.8.2)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.9/dist-packages (from pandas->llama-index) (2022.7.1)\n",
"Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.9/dist-packages (from tiktoken->llama-index) (2022.6.2)\n",
"Collecting requests>=2.20\n",
" Downloading requests-2.28.2-py3-none-any.whl (62 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.8/62.8 KB\u001b[0m \u001b[31m4.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting yarl<2.0,>=1.0\n",
" Downloading yarl-1.8.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m264.6/264.6 KB\u001b[0m \u001b[31m13.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting charset-normalizer<4.0,>=2.0\n",
" Downloading charset_normalizer-3.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (199 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m199.2/199.2 KB\u001b[0m \u001b[31m10.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai>=0.26.4->llama-index) (22.2.0)\n",
"Collecting async-timeout<5.0,>=4.0.0a3\n",
" Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n",
"Collecting frozenlist>=1.1.1\n",
" Downloading frozenlist-1.3.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m158.8/158.8 KB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting multidict<7.0,>=4.5\n",
" Downloading multidict-6.0.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m114.2/114.2 KB\u001b[0m \u001b[31m4.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting aiosignal>=1.1.2\n",
" Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)\n",
"Requirement already satisfied: packaging>=17.0 in /usr/local/lib/python3.9/dist-packages (from marshmallow<4.0.0,>=3.3.0->dataclasses_json->llama-index) (23.0)\n",
"Requirement already satisfied: typing-extensions>=4.2.0 in /usr/local/lib/python3.9/dist-packages (from pydantic<2,>=1->langchain->llama-index) (4.5.0)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/dist-packages (from python-dateutil>=2.8.1->pandas->llama-index) (1.15.0)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai>=0.26.4->llama-index) (2022.12.7)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai>=0.26.4->llama-index) (1.26.15)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai>=0.26.4->llama-index) (2.10)\n",
"Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.9/dist-packages (from SQLAlchemy<2,>=1->langchain->llama-index) (2.0.2)\n",
"Collecting mypy-extensions>=0.3.0\n",
" Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
"Building wheels for collected packages: llama-index\n",
" Building wheel for llama-index (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for llama-index: filename=llama_index-0.4.29-py3-none-any.whl size=207576 sha256=1d3f598d77ac89d4da43ce240fed39c0bf7c6bdb05055f2be627635642987e65\n",
" Stored in directory: /root/.cache/pip/wheels/7e/3a/11/04d7ff1568bcf200ddf1c7124589ac943a4061c7de040448d4\n",
"Successfully built llama-index\n",
"Installing collected packages: mypy-extensions, multidict, marshmallow, frozenlist, charset-normalizer, async-timeout, yarl, typing-inspect, requests, marshmallow-enum, aiosignal, tiktoken, dataclasses_json, aiohttp, openai, langchain, llama-index\n",
" Attempting uninstall: requests\n",
" Found existing installation: requests 2.25.1\n",
" Uninstalling requests-2.25.1:\n",
" Successfully uninstalled requests-2.25.1\n",
"Successfully installed aiohttp-3.8.4 aiosignal-1.3.1 async-timeout-4.0.2 charset-normalizer-3.1.0 dataclasses_json-0.5.7 frozenlist-1.3.3 langchain-0.0.113 llama-index-0.4.29 marshmallow-3.19.0 marshmallow-enum-1.5.1 multidict-6.0.4 mypy-extensions-1.0.0 openai-0.27.2 requests-2.28.2 tiktoken-0.3.1 typing-inspect-0.8.0 yarl-1.8.2\n",
"Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
"Requirement already satisfied: openai in /usr/local/lib/python3.9/dist-packages (0.27.2)\n",
"Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from openai) (4.65.0)\n",
"Requirement already satisfied: aiohttp in /usr/local/lib/python3.9/dist-packages (from openai) (3.8.4)\n",
"Requirement already satisfied: requests>=2.20 in /usr/local/lib/python3.9/dist-packages (from openai) (2.28.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai) (2.10)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai) (2022.12.7)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai) (1.26.15)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.9/dist-packages (from requests>=2.20->openai) (3.1.0)\n",
"Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (4.0.2)\n",
"Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (6.0.4)\n",
"Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (22.2.0)\n",
"Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (1.8.2)\n",
"Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (1.3.1)\n",
"Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.9/dist-packages (from aiohttp->openai) (1.3.3)\n"
]
}
],
"source": [
"!pip install llama-index\n",
"!pip install openai"
]
},
{
"cell_type": "code",
"source": [
"# Import necessary packages\n",
"from llama_index import GPTSimpleVectorIndex, Document, SimpleDirectoryReader\n",
"import os"
],
"metadata": {
"id": "UWLAePPKbZK4"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"os.environ['OPENAI_API_KEY'] = 'API KEY di Scienza in rete'\n"
],
"metadata": {
"id": "swTddxmibloL"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"loader = DocxReader()\n",
"documents = loader.load_data(file=Path('/content/La sfida delle materie prime critiche per la transizione energetica.docx'))"
],
"metadata": {
"id": "XMa1HaCmd4Ti"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"source": [
"index = GPTSimpleVectorIndex(documents)"
],
"metadata": {
"id": "1D9w3h-LdIlK"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Interroga Chat-gpt sull'articolo [La sfida delle materie prime critiche per la transizione energetica](https://www.scienzainrete.it/articolo/sfida-delle-materie-prime-critiche-transizione-energetica/riccardo-lo-bue/2023-03-13). Fai click sulla riga 60 e modifica la domanda, poi fai click su tasto esegui."
],
"metadata": {
"id": "QU2UB4ttOLTs"
}
},
{
"cell_type": "code",
"source": [
"response = index.query(\"A cosa servono le materie prime critiche?\")\n",
"print(response)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "P7dNPxLTeGQC",
"outputId": "1ac48557-8a7c-47ed-e3fd-9fbeb5b6d1e1"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
"Le materie prime critiche sono materiali di importanza strategica dal punto di vista economico, necessari per la realizzazione di turbine eoliche, batterie per veicoli elettrici, reti elettriche e altre infrastrutture necessarie per la transizione energetica sostenibile sul territorio comunitario. Recentemente, proprio grazie a tecniche sostenibili di prospezione mineraria, è stato scoperto un giacimento di terre rare in Svezia, le cui riserve ammontano a oltre un milione di tonnellate. Inoltre, le materie prime critiche servono anche a sviluppare partenariati strategici che consolidino le relazioni commerciali con Paesi terzi ricchi di materie prime critiche. In particolare, la UE potrebbe beneficiare dagli accordi bilaterali ed è già impegnata in diversi tavoli internazionali focalizzati sulle materie prime critiche. In particol\n"
]
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment