Use GPT-4 to automatically diagnose errors

_result.md

TLDR

I wrapped Python traceback and Exception so whenever an Exception is raised, an automatic diagnosis is done by GPT-4. See results below:

Prompt (Automatically Generated by error_wrap function)

This error message occurred because of 'test' at line 177 in file api.py, function main. The exception type is <class 'Exception'>. The function is:

async def main():
    try:
       raise Exception("test")
    except Exception as e:
        potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
        print (potential_fix)
        return {"error": str(e), "potential_fix": potential_fix}

How do I fix this?

GPT-4 Completion

The error message is occurring because you are deliberately raising an Exception with the message "test". To fix this, you need to remove or comment out the line where the exception is being raised.

Here's the modified code without the exception being raised:

async def main():
    try:
       # remove or comment out the following line
       # raise Exception("test")
       # add your actual code here
    except Exception as e:
        potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
        print (potential_fix)
        return {"error": str(e), "potential_fix": potential_fix}

If you want to test your error handling without actually removing the line, you can simply comment it out, as shown above. Add your actual code inside the try block, and the error handling will work as intended.

P.S.

This might also work in Javascript/Typescript but requires more steps (https://chat.openai.com/share/ca1ad6ad-68f6-4494-a35b-b1514deca12c)

error_wrap.py

import openai
import os
import dotenv
import json
import sys
import inspect

# read these for more info:
# https://chat.openai.com/share/9341df1e-65c0-4fd8-87dc-80e0dc9fa5bc
# https://chat.openai.com/share/193482c6-e7b3-4022-b42b-cd2530efb839

dotenv.load_dotenv()

openai.api_key = os.getenv("OPENAI_API_KEY")

async def run_chat_prompt(prompt, model="gpt-4"):
    completion = openai.ChatCompletion.create(
    model=model,
    messages=[
        {"role": "user", "content": prompt}
    ]
    )

    return (completion.choices[0].message.content)

def wrap_error(e: Exception, description: str = None):
    exc_type, exc_value, exc_traceback = sys.exc_info()
    filename = exc_traceback.tb_frame.f_code.co_filename
    line_number = exc_traceback.tb_lineno
    func_name = exc_traceback.tb_frame.f_code.co_name
    module = inspect.getmodule(exc_traceback.tb_frame)
    function_obj = getattr(module, func_name)

    # Get and print the source code of the function
    func_code = inspect.getsource(function_obj)

    error_message = f"This error message occurred because of '{str(e)}' at line {line_number} in file {filename}, function {func_name}. The exception type is {exc_type}. The function is: \n\n```python\n{func_code}\n```\n\n How do I fix this?"
    if description:
        error_message += f'Other details: {description}'

    return error_message

main.py

import error_wrap

async def main():
    try:
       raise Exception("test")
    except Exception as e:
        potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
        # an optional description can be added with something along the lines of: error_wrap.wrap_error(e, "here's some specific info passed in by me, a human")
        return {"error": str(e), "potential_fix": potential_fix}

Great idea man, thank you. I took it a step further and make it a decorator, so now all I do is @gpt_error and decorate all my functions and it prints the responses to the console.

Thanks for the idea my friend.

Here is my contribution thanks to your code. It handles async or non async functions and now supports azure and openai. I should prob wrap the openai call in some error handling, but....You all can do that ;)

import asyncio
import openai
import sys
import traceback
import inspect
from functools import wraps
import termcolor

openai.api_type = "azure"
openai.api_base = os.environ.get("AZURE_URL")
openai.api_version = os.environ.get("API_VERSION")
openai.api_key = os.environ.get("AZURE_API_KEY")
engine = os.environ.get("AZURE_MODEL_NAME")

def gpt_error(func):
    if inspect.iscoroutinefunction(func):
        @wraps(func)
        async def async_wrapper(*args, **kwargs):
            try:
                return await func(*args, **kwargs)
            except Exception as e:
                return await handle_error(e, func)
        return async_wrapper
    else:
        @wraps(func)
        def sync_wrapper(*args, **kwargs):
            try:
                return func(*args, **kwargs)
            except Exception as e:
                print(termcolor.colored(f"Error: {e}", color="red"))
                error_message = asyncio.run(handle_error(e, func))
                print(error_message)
                return error_message
        return sync_wrapper

async def handle_error(e, func):
    exc_type, exc_value, exc_traceback = sys.exc_info()
    stack_trace = traceback.extract_tb(exc_traceback)
    last_trace = stack_trace[-1]
    filename = last_trace[0]
    line_number = last_trace[1]
    func_name = last_trace[2]

    # Get and print the source code of the function
    func_code = inspect.getsource(func)

    error_message = f"This error message occurred because of '{str(e)}' at line {line_number} in file {filename}, function {func_name}. The exception type is {exc_type}. The function is: \n\n```python\n{func_code}\n```\n\n Provide a detailed explanation on why this error occurred and sample python code on how to rectify the problem."

    # Make an API call to OpenAI chat with the error message
    completion = await openai.ChatCompletion.acreate(
        engine="chatGPT",
        messages=[
            {"role": "user", "content": error_message}
        ]
    )

    
    return completion.choices[0].message.content

I took it a step further and make it a decorator, so now all I do is @gpt_error and decorate all my functions and it prints the responses to the console.

@Cytranics that's an amazing idea! I didn't think of using decorators but that is definitely the way to do it. Thank you for the improvement! I'm decorating my codebase with it as we speak!! :)

Hello Sir,
your detailed Readme of the Bramses' Highly Opinionated Vault 2023 got my attention.
I would love to learn more on your thoughts and ideas.
It is probably due to me being not that experienced in the use of github, but I really could not find any way to download a zipped version of your vault that I could use.
Hopefully you can help, but anyway I wish you a lot of success with your adventures.
I subscribed to your newsletter.
All the best,
Wim Claessen
Netherlands.