few-shot-learning-on-function-calling.ipynb

few-shot-learning-on-function-calling.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/kyo-takano/7bf43331b79c6c6678457024ff8d412a/few-shot-learning-on-function-calling.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Few-Shot Learning on Function Calling\n",
        "\n",
        "This notebook uses [the official example by OpenAI](https://platform.openai.com/docs/guides/text-generation/function-calling) to demonstrate how you can construct a few-shot learning prompt.\n",
        "\n",
        "---\n",
        "\n",
        "**Last Updated**: December 29, 2023 — Newly added implementations compatible with `openai==1.*`.\n",
        "\n",
        "With the major upgrade of `openai` SDK from version `0.*` to `1.*`, **the way we can implement one-shot/few-shot learning has also changed**."
      ],
      "metadata": {
        "id": "VSg8cgC7Eket"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## `openai==1.*`\n",
        "\n",
        "As far as we could figure out, there are two ways you can do few-shot learning with the new `openai` SDK."
      ],
      "metadata": {
        "id": "tTmTzRZW5pSl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q openai==1.*"
      ],
      "metadata": {
        "id": "Iqs2F7xX5tKQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Using `tool_call`\n",
        "\n",
        "To successfully insert few-shot exemplars into a chat prompt, you need to do the following things:\n",
        "\n",
        "1. Make up a random ID starting with \"call_\" (e.g., `call_qwertyuiop`); it can be pretty much anything\n",
        "2. Append ***2*** dictionaries per example output linked by the random ID (new from v1.0)\n",
        "\n",
        "Remember that the way you define function(s) and force the call is also different from `function_call`."
      ],
      "metadata": {
        "id": "NE5BDG3J8HH0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "from openai import OpenAI\n",
        "client = OpenAI()\n",
        "\n",
        "tool = {\n",
        "    \"type\": \"function\",\n",
        "    \"function\": {\n",
        "        \"name\": \"get_current_weather\",\n",
        "        \"description\": \"Get the current weather in a given location\",\n",
        "        \"parameters\": {\n",
        "            \"type\": \"object\",\n",
        "            \"properties\": {\n",
        "                \"location\": {\n",
        "                    \"type\": \"string\",\n",
        "                    \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
        "                },\n",
        "                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
        "            },\n",
        "            \"required\": [\"location\"],\n",
        "        },\n",
        "    }\n",
        "}\n",
        "\n",
        "call_id = 'call_qwertyuiop'\n",
        "\n",
        "exemplar_in = \"Is it raining in Tokyo today?\"\n",
        "exemplar_out = {\n",
        "    \"location\":\"Tokyo\",\n",
        "    \"unit\":\"celsius\"\n",
        "}\n",
        "\n",
        "messages = [\n",
        "    {'role': 'user', 'content': exemplar_in},\n",
        "    {\n",
        "        'role': 'assistant',\n",
        "        'tool_calls': [{\n",
        "            'id': call_id,\n",
        "            'function': {\n",
        "                'name': tool[\"function\"][\"name\"],\n",
        "                'arguments': json.dumps(exemplar_out)\n",
        "            },\n",
        "            'type': 'function'\n",
        "        }],\n",
        "    },\n",
        "    {\n",
        "        \"role\": \"tool\",\n",
        "        \"tool_call_id\": call_id,\n",
        "        \"name\": tool[\"function\"][\"name\"],\n",
        "        \"content\": json.dumps(exemplar_out),\n",
        "    },\n",
        "    {'role': 'user', 'content': \"What's the current weather in Rome, Italy.\"},\n",
        "]\n",
        "response = client.chat.completions.create(\n",
        "    model=\"gpt-3.5-turbo-1106\",\n",
        "    messages=messages,\n",
        "    tools=[tool],\n",
        "    # Force calling the function as a tool:\n",
        "    tool_choice={\n",
        "        \"type\": \"function\",\n",
        "        \"function\": {\n",
        "            \"name\": tool[\"function\"][\"name\"]\n",
        "        }\n",
        "    },\n",
        ")\n",
        "response_message = response.choices[0].message\n",
        "response_message.model_dump()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VfxAKGI68LOC",
        "outputId": "bb6b7c6a-c7e5-4a0e-96f0-fbdf1745233e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'content': None,\n",
              " 'role': 'assistant',\n",
              " 'function_call': None,\n",
              " 'tool_calls': [{'id': 'call_RzHoNr5LfuTXA0FLH7db7ZrH',\n",
              "   'function': {'arguments': '{\"location\":\"Rome, Italy\",\"unit\":\"celsius\"}',\n",
              "    'name': 'get_current_weather'},\n",
              "   'type': 'function'}]}"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Using `function_call` (*deprecated*)\n",
        "\n",
        "You can also keep using the well-familiar `function_call` instead of `tool_call`.\n",
        "\n",
        "This approach is much simpler and easier, although it is [labeled as *deprecated* in the documentation](https://platform.openai.com/docs/api-reference/chat/create#chat-create-function_call)."
      ],
      "metadata": {
        "id": "_r7tjkvC6KrX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from openai import OpenAI\n",
        "client = OpenAI()\n",
        "\n",
        "function = {\n",
        "    \"name\": \"get_current_weather\",\n",
        "    \"description\": \"Get the current weather in a given location\",\n",
        "    \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "            \"location\": {\n",
        "                \"type\": \"string\",\n",
        "                \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
        "            },\n",
        "            \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
        "        },\n",
        "        \"required\": [\"location\"],\n",
        "    },\n",
        "}\n",
        "\n",
        "messages = [\n",
        "    {'role': 'user', 'content': \"Is it raining in Tokyo today?\"},\n",
        "    {\n",
        "        'role': 'assistant',\n",
        "        'function_call': {\n",
        "            'arguments': '{\"location\":\"Tokyo\", \"unit\":\"celsius\"}',\n",
        "            'name': 'get_current_weather'\n",
        "        },\n",
        "    },\n",
        "    {'role': 'user', 'content': \"What's the current weather in Rome, Italy.\"},\n",
        "]\n",
        "response = client.chat.completions.create(\n",
        "    model=\"gpt-3.5-turbo-1106\",\n",
        "    messages=messages,\n",
        "    functions=[function],\n",
        "    function_call={\"name\": function[\"name\"]}, # this forces calling `function`\n",
        ")\n",
        "response_message = response.choices[0].message\n",
        "response_message.model_dump()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Pcv0nIPE7TgP",
        "outputId": "3157fe6d-a33d-4e1f-d2bb-aca4b87f1184"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'content': None,\n",
              " 'role': 'assistant',\n",
              " 'function_call': {'arguments': '{\"location\":\"Rome, Italy\",\"unit\":\"celsius\"}',\n",
              "  'name': 'get_current_weather'},\n",
              " 'tool_calls': None}"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## `openai<1.0.0` (`0.27.0~0.28.1`)\n",
        "\n",
        "Below is the legacy implementation of few-shot learning with earlier versions (`0.27.0` to `0.28.1`). We retain this information for future reference and comparison."
      ],
      "metadata": {
        "id": "6y-awKZQ18lo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -q \"openai<1.0,>=0.27.0\""
      ],
      "metadata": {
        "id": "YCjyR2nHOi9i"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "OpenAI API returns an object similar to the one below when a function call is recommended:\n",
        "\n",
        "```python\n",
        ">>> response = openai.ChatCompletion.create(\n",
        "...     model=\"gpt-3.5-turbo-0613\",\n",
        "...     messages=messages,\n",
        "...     functions=functions,\n",
        "... )\n",
        ">>> response_message = response[\"choices\"][0][\"message\"]\n",
        ">>> print(response_message)\n",
        "<OpenAIObject at 0x000000000000> JSON: {\n",
        "  \"role\": \"assistant\",\n",
        "  \"content\": null,\n",
        "  \"function_call\": {\n",
        "    \"name\": \"get_current_weather\",\n",
        "    \"arguments\": \"{\\n  \\\"location\\\": \\\"Rome\\\",\\n  \\\"unit\\\": \\\"celsius\\\"\\n}\"\n",
        "  }\n",
        "}\n",
        "```\n",
        "\n",
        "I found a way to implement **few-shot learning based on function calling** by constructing this `OpenAIObject` with the necessary properties.\n",
        "\n",
        "While I haven't seen anyone else using this technique, I find it particularly useful when estimating the arguments is a challenging task in itself (e.g., information extraction), or when the target function requires strict formatting and typing."
      ],
      "metadata": {
        "id": "ZjyW_Do7ApqZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Implementation\n",
        "\n",
        "I demonstrate how to construct an `OpenAIObject` suggesting a function call as a one-shot target.\n",
        "\n",
        "First, I define a function to call."
      ],
      "metadata": {
        "id": "u2AkkluxE2p3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "function = {\n",
        "    \"name\": \"get_current_weather\",\n",
        "    \"description\": \"Get the current weather in a given location\",\n",
        "    \"parameters\": {\n",
        "        \"type\": \"object\",\n",
        "        \"properties\": {\n",
        "            \"location\": {\n",
        "                \"type\": \"string\",\n",
        "                \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
        "            },\n",
        "            \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
        "        },\n",
        "        \"required\": [\"location\"],\n",
        "    },\n",
        "}"
      ],
      "metadata": {
        "id": "BlUL6VBV9umG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, I create the corresponding `OpenAIObject`."
      ],
      "metadata": {
        "id": "8gO7Zp1LGiVe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import json\n",
        "import openai\n",
        "\n",
        "example = openai.openai_object.OpenAIObject()\n",
        "example[\"role\"] = \"assistant\"\n",
        "example[\"content\"] = None\n",
        "example[\"function_call\"] = {\n",
        "    \"name\": function[\"name\"],\n",
        "    \"arguments\": json.dumps(\n",
        "        {\n",
        "            \"location\": \"Rome\",\n",
        "            \"unit\": \"celsius\",\n",
        "        },\n",
        "        indent=2,\n",
        "        ensure_ascii=False,\n",
        "    ),\n",
        "}\n",
        "example"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "o7x-FvnHAf34",
        "outputId": "de81edff-8801-4d19-94c3-e323b2807f42"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<OpenAIObject at 0x7bb7862f0b80> JSON: {\n",
              "  \"role\": \"assistant\",\n",
              "  \"content\": null,\n",
              "  \"function_call\": {\n",
              "    \"name\": \"get_current_weather\",\n",
              "    \"arguments\": \"{\\n  \\\"location\\\": \\\"Rome\\\",\\n  \\\"unit\\\": \\\"celsius\\\"\\n}\"\n",
              "  }\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Note that `json.dumps` with `indent=2` and `ensure_ascii=False` provides the correct formatting as if returned by the API.\n",
        "\n",
        "Finally, you can make a call with the above being a one-shot.\n"
      ],
      "metadata": {
        "id": "zuorKzBhGrK5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "messages = [\n",
        "    {'role': 'user', 'content': \"What's the current weather in Rome, Italy.\"},\n",
        "    example,\n",
        "    {'role': 'user', 'content': \"Is it raining in Tokyo today?\"},\n",
        "]\n",
        "response = openai.ChatCompletion.create(\n",
        "    model=\"gpt-3.5-turbo-0613\",\n",
        "    messages=messages,\n",
        "    functions=[function],\n",
        "    function_call={\"name\": function[\"name\"]}, # this forces calling `function`\n",
        ")\n",
        "response_message = response[\"choices\"][0][\"message\"]\n",
        "response_message"
      ],
      "metadata": {
        "id": "vvaPSVKtAc-a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d8ec8fbf-2be3-495e-9429-127dba0f99f6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<OpenAIObject at 0x7bb77489b9c0> JSON: {\n",
              "  \"role\": \"assistant\",\n",
              "  \"content\": null,\n",
              "  \"function_call\": {\n",
              "    \"name\": \"get_current_weather\",\n",
              "    \"arguments\": \"{\\n  \\\"location\\\": \\\"Tokyo\\\",\\n  \\\"unit\\\": \\\"celsius\\\"\\n}\"\n",
              "  }\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "While I have demonstrated it with an extremely simple task, this technique would be especially useful for more complex tasks."
      ],
      "metadata": {
        "id": "UWoKT8GaHd2q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Failure"
      ],
      "metadata": {
        "id": "NF0OnXrAEdjA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "As a side note, instantiating `OpenAIObject` like below does not work."
      ],
      "metadata": {
        "id": "03_n4t-UIAMm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "example_wrong = openai.openai_object.OpenAIObject({\n",
        "    \"role\": \"assistant\",\n",
        "    \"content\": None,\n",
        "    \"function_call\": json.dumps(\n",
        "        {\n",
        "            \"name\": function[\"name\"],\n",
        "            \"arguments\": {\n",
        "                \"location\": \"Rome\",\n",
        "                \"unit\": \"celsius\",\n",
        "            }\n",
        "        },\n",
        "        indent=2,\n",
        "        ensure_ascii=False\n",
        "    ),\n",
        "})\n",
        "example_wrong"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ws93KZdU9v4M",
        "outputId": "9544cb50-6f3d-47d9-e29b-c995542885b7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<OpenAIObject at 0x7bb7532899e0> JSON: {\n",
              "  \"id\": {\n",
              "    \"role\": \"assistant\",\n",
              "    \"content\": null,\n",
              "    \"function_call\": \"{\\n  \\\"name\\\": \\\"get_current_weather\\\",\\n  \\\"arguments\\\": {\\n    \\\"location\\\": \\\"Rome\\\",\\n    \\\"unit\\\": \\\"celsius\\\"\\n  }\\n}\"\n",
              "  }\n",
              "}"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This is because the resulting OpenAIObject includes an undesired \"id\" key at the top of its JSON data."
      ],
      "metadata": {
        "id": "jYVaB7iVINGT"
      }
    }
  ]
}

Is this still the best way of doing it? 29/10/2023

This technically still works but beware that function calls are outdated since now Openai has switched to tools. Same principle here goes but syntax has changed