agentic_reports.ipynb

agentic_reports.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPJLj5Z57McuCqtT8ZPz9zC",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/ruvnet/cf6fc998c0c721f85543f78eb1a1a169/agentic_reports.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 📄 Agentic Report Generator\n",
        "\n",
        "Welcome to the Agentic Report Generator! This tool leverages the power of OpenAI's GPT-4 and Exa's search capabilities to generate comprehensive research reports on any topic you choose.\n",
        "\n",
        "## How It Works\n",
        "1. **Generate Subqueries**: The tool will create several subqueries based on your main topic to ensure a thorough investigation.\n",
        "2. **Search with Exa**: Each subquery is searched using Exa to gather the most relevant and recent information.\n",
        "3. **Format Results**: The search results are formatted for easy consumption by the language model.\n",
        "4. **Generate Report**: A professional, three-paragraph research report is generated, complete with citations.\n",
        "\n",
        "## Getting Started\n",
        "1. **Add Your API Keys**:\n",
        "   - **Exa API Key**: Obtain your API key from [Exa.ai](https://exa.ai) and store it securely in Google Colab's Secrets.\n",
        "   - **OpenAI API Key**: Obtain your API key from [OpenAI](https://openai.com) and store it securely in Google Colab's Secrets.\n",
        "2. **Run the Cells**: Execute the cells in the notebook to start generating your report.\n",
        "\n",
        "Let's get started!"
      ],
      "metadata": {
        "id": "z8fsb44MMHfF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# @title Install Requirements\n",
        "!pip install openai exa_py"
      ],
      "metadata": {
        "id": "-LKUeN72Imim",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_49IDO56IV6M",
        "outputId": "5f2e861f-7909-48e2-a897-e3ec0f0d445f"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Welcome to the Research Report Generator!\n",
            "Please select an option:\n",
            "1. Generate a report\n",
            "2. Exit\n",
            "Enter your choice: 1\n",
            "Please enter the topic you want to research: latest stories\n",
            "Starting report generation for topic: latest stories\n",
            " \n",
            "🌿 Generating subqueries from topic: latest stories\n",
            "\n",
            "⌛ Searching each subquery\n",
            "Generating report from Exa results for topic: latest stories\n",
            "\n",
            "⌨️  Formatting Exa results for LLM\n",
            "📃 Final Report: ## Research Report\n",
            "\n",
            "In recent developments in Alabama's parole system, there has been a notable decision involving Kenneth Wayne McCroskey, who was initially imprisoned for a third-degree robbery in 1996. McCroskey, despite serving a life sentence for the violation of parole terms related to his previous offenses, was paroled on Wednesday after the state revisited his case [1]. Alabama's parole system has faced criticism for being overly stringent, with McCroskey's case shedding light on its severity. The present parole board's increased rate of approvals might suggest a potential shift in the system, indicating a move towards more leniency and mercy in certain cases.\n",
            "\n",
            "The tech industry recently observed a significant ethical lapse regarding AI in journalism. Hoodline, a local news site, has supplemented its reporting with AI-generated content, posing significant questions about transparency and credibility in journalism. While AI can assist in handling large data sets and facilitate research, Hoodline faced serious backlash for using fake biographies and AI-generated headshots to give an illusion of human-authored content. This controversial move underscores the media's struggle with ethical AI usage, particularly as it risks public trust and exacerbates misinformation [2]. Despite employing multiple editors to oversee the AI's contributions, the integrity of AI-generated content remains a contentious issue.\n",
            "\n",
            "A landmark incident in American history occurred when former President Donald J. Trump was convicted of multiple felony counts related to falsifying business records. This conviction arises from his efforts to cover up a scandal that potentially threatened his 2016 presidential campaign. Trump's case is set against the backdrop of an electorate that remains deeply divided; however, this conviction could significantly impact his political future and the wider politico-legal landscape in the United States [3]. The impending sentence, which could range from probation to a potential four-year prison term, exemplifies the judiciary's potent role in upholding legal accountability for even the country's highest officeholders.\n",
            "\n",
            "## References\n",
            "[1] <https://www.al.com/news/huntsville/2024/05/he-missed-a-meeting-and-got-sent-back-to-prison-now-alabama-is-giving-him-another-chance.html>, May 2024.\n",
            "[2] <https://edition.cnn.com/2024/05/30/media/ai-bylines-local-news-hoodline/index.html>, May 2024.\n",
            "[3] <https://www.cnbc.com/2024/05/30/trump-trial-verdict-hush-money.html>, May 2024.\n",
            "Would you like to ask a follow-up question? (yes/no): yes\n",
            "Please enter your follow-up question: test\n",
            "Follow-up question: test\n",
            "Starting report generation for topic: test\n",
            " \n",
            "🌿 Generating subqueries from topic: test\n",
            "\n",
            "⌛ Searching each subquery\n",
            "Generating report from Exa results for topic: test\n",
            "\n",
            "⌨️  Formatting Exa results for LLM\n"
          ]
        }
      ],
      "source": [
        "# @title 📄 Agentic Report Generator\n",
        "import os\n",
        "import json\n",
        "import openai\n",
        "from exa_py import Exa\n",
        "from datetime import datetime, timedelta\n",
        "from google.colab import userdata\n",
        "\n",
        "# Access secrets stored in Google Colab\n",
        "EXA_API_KEY = userdata.get('EXA_API_KEY')\n",
        "OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')\n",
        "\n",
        "# Set up API keys using environment variables\n",
        "os.environ['EXA_API_KEY'] = EXA_API_KEY\n",
        "os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY\n",
        "\n",
        "# Set up API keys\n",
        "openai.api_key = OPENAI_API_KEY\n",
        "exa = Exa(api_key=EXA_API_KEY)\n",
        "\n",
        "def display_menu():\n",
        "    print(\"Welcome to the Research Report Generator!\")\n",
        "    print(\"Please select an option:\")\n",
        "    print(\"1. Generate a report\")\n",
        "    print(\"2. Exit\")\n",
        "\n",
        "def get_user_input(prompt):\n",
        "    return input(prompt)\n",
        "\n",
        "def generate_report_without_exa(topic):\n",
        "    print(f\"🚀 Generating report without Exa for topic: {topic}\")\n",
        "    print(f\"\")\n",
        "    content = f\"Write a comprehensive and professional three paragraph research report about {topic}. Include citations with source, month, and year.\"\n",
        "    completion = openai.chat.completions.create(\n",
        "        model='gpt-4o',\n",
        "        messages=[{\"role\": \"user\", \"content\": content}],\n",
        "        temperature=0\n",
        "    )\n",
        "    report = completion.choices[0].message.content\n",
        "    return report\n",
        "\n",
        "def create_custom_function(num_subqueries):\n",
        "    properties = {}\n",
        "    for i in range(1, num_subqueries + 1):\n",
        "        key = f'subquery_{i}'\n",
        "        properties[key] = {\n",
        "            'type': 'string',\n",
        "            'description': 'Search queries that would be useful for generating a report on my main topic'\n",
        "        }\n",
        "\n",
        "    custom_function = {\n",
        "        'name': 'generate_exa_search_queries',\n",
        "        'description': 'Generates Exa search queries to investigate the main topic',\n",
        "        'parameters': {\n",
        "            'type': 'object',\n",
        "            'properties': properties\n",
        "        }\n",
        "    }\n",
        "\n",
        "    return [custom_function]\n",
        "\n",
        "def generate_subqueries_from_topic(topic, num_subqueries=6):\n",
        "    print(f\" \")\n",
        "    print(f\"🌿 Generating subqueries from topic: {topic}\")\n",
        "    content = f\"I'm going to give you a topic I want to research. I want you to generate {num_subqueries} interesting, diverse search queries that would be useful for generating a report on my main topic. Here is the main topic: {topic}.\"\n",
        "    custom_functions = create_custom_function(num_subqueries)\n",
        "    completion = openai.chat.completions.create(\n",
        "        model='gpt-4o',\n",
        "        messages=[{\"role\": \"user\", \"content\": content}],\n",
        "        temperature=0,\n",
        "        functions=custom_functions,\n",
        "        function_call='auto'\n",
        "    )\n",
        "    json_response = json.loads(completion.choices[0].message.function_call.arguments)\n",
        "    subqueries = list(json_response.values())\n",
        "    return subqueries\n",
        "\n",
        "def exa_search_each_subquery(subqueries):\n",
        "    print(f\"\")\n",
        "    print(f\"⌛ Searching each subquery\")\n",
        "    list_of_query_exa_pairs = []\n",
        "    one_week_ago = (datetime.now() - timedelta(days=7)).strftime(\"%Y-%m-%d\")\n",
        "    for query in subqueries:\n",
        "        search_response = exa.search_and_contents(\n",
        "            query,\n",
        "            num_results=5,\n",
        "            use_autoprompt=True,\n",
        "            start_published_date=one_week_ago,\n",
        "            text=True,\n",
        "            highlights={\"num_sentences\": 5},\n",
        "        )\n",
        "        query_object = {\n",
        "            'subquery': query,\n",
        "            'results': search_response.results\n",
        "        }\n",
        "        list_of_query_exa_pairs.append(query_object)\n",
        "    return list_of_query_exa_pairs\n",
        "\n",
        "def format_exa_results_for_llm(list_of_query_exa_pairs):\n",
        "    print(f\"\")\n",
        "    print(f\"⌨️  Formatting Exa results for LLM\")\n",
        "    formatted_string = \"\"\n",
        "    for i in list_of_query_exa_pairs:\n",
        "        formatted_string += f\"[{i['subquery']}]:\\n\"\n",
        "        for result in i['results']:\n",
        "            content = result.text if result.text else \" \".join(result.highlights)\n",
        "            publish_date = result.published_date\n",
        "            formatted_string += f\"URL: {result.url}\\nContent: {content}\\nPublish Date: {publish_date}\\n\"\n",
        "        formatted_string += \"\\n\"\n",
        "    return formatted_string\n",
        "\n",
        "def generate_report_from_exa_results(topic, list_of_query_exa_pairs):\n",
        "    print(f\"Generating report from Exa results for topic: {topic}\")\n",
        "    formatted_exa_content = format_exa_results_for_llm(list_of_query_exa_pairs)\n",
        "    content = (f\"Write a comprehensive and professional three paragraph research report about {topic} based on the provided information. \"\n",
        "               f\"Include citations in the text using footnote notation ([citation #]), for example [2]. First provide the report, followed by a single `References` section \"\n",
        "               f\"that only lists the URLs (and their published date) used, in the format [#] <url>. For the published date, only include the month and year. \"\n",
        "               f\"Reset the citations index and ignore the order of citations in the provided information. Here is the information: {formatted_exa_content}.\")\n",
        "    completion = openai.chat.completions.create(\n",
        "        model='gpt-4o',\n",
        "        messages=[{\"role\": \"user\", \"content\": content}]\n",
        "    )\n",
        "    report = completion.choices[0].message.content\n",
        "    return report\n",
        "\n",
        "def generate_report(topic):\n",
        "    print(f\"Starting report generation for topic: {topic}\")\n",
        "    subqueries = generate_subqueries_from_topic(topic)\n",
        "    list_of_query_exa_pairs = exa_search_each_subquery(subqueries)\n",
        "    report = generate_report_from_exa_results(topic, list_of_query_exa_pairs)\n",
        "    return report\n",
        "\n",
        "def main():\n",
        "    while True:\n",
        "        display_menu()\n",
        "        choice = get_user_input(\"Enter your choice: \")\n",
        "        if choice == '1':\n",
        "            example_topic = get_user_input(\"Please enter the topic you want to research: \")\n",
        "            report = generate_report(example_topic)\n",
        "            print(f\"📃 Final Report: {report}\")\n",
        "            while True:\n",
        "                follow_up = get_user_input(\"Would you like to ask a follow-up question? (yes/no): \")\n",
        "                if follow_up.lower() == 'yes':\n",
        "                    follow_up_question = get_user_input(\"Please enter your follow-up question: \")\n",
        "                    print(f\"Follow-up question: {follow_up_question}\")\n",
        "                    report = generate_report(follow_up_question)\n",
        "                    print(f\"📃 Final Report: {report}\")\n",
        "                else:\n",
        "                    print(\"Thank you for using the Research Report Generator!\")\n",
        "                    break\n",
        "        elif choice == '2':\n",
        "            print(\"Exiting the program. Goodbye!\")\n",
        "            break\n",
        "        else:\n",
        "            print(\"Invalid choice. Please try again.\")\n",
        "\n",
        "\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    main()\n"
      ]
    }
  ]
}