Skip to content

Instantly share code, notes, and snippets.

@bramses
Last active January 8, 2024 09:45
Show Gist options
  • Save bramses/3bc23c6c601e7ad5b250c9f5054870d5 to your computer and use it in GitHub Desktop.
Save bramses/3bc23c6c601e7ad5b250c9f5054870d5 to your computer and use it in GitHub Desktop.
Use GPT-4 to automatically diagnose errors

TLDR

I wrapped Python traceback and Exception so whenever an Exception is raised, an automatic diagnosis is done by GPT-4. See results below:

Prompt (Automatically Generated by error_wrap function)

This error message occurred because of 'test' at line 177 in file api.py, function main. The exception type is <class 'Exception'>. The function is:

async def main():
    try:
       raise Exception("test")
    except Exception as e:
        potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
        print (potential_fix)
        return {"error": str(e), "potential_fix": potential_fix}

How do I fix this?

GPT-4 Completion

The error message is occurring because you are deliberately raising an Exception with the message "test". To fix this, you need to remove or comment out the line where the exception is being raised.

Here's the modified code without the exception being raised:

async def main():
    try:
       # remove or comment out the following line
       # raise Exception("test")
       # add your actual code here
    except Exception as e:
        potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
        print (potential_fix)
        return {"error": str(e), "potential_fix": potential_fix}

If you want to test your error handling without actually removing the line, you can simply comment it out, as shown above. Add your actual code inside the try block, and the error handling will work as intended.

P.S.

This might also work in Javascript/Typescript but requires more steps (https://chat.openai.com/share/ca1ad6ad-68f6-4494-a35b-b1514deca12c)

import openai
import os
import dotenv
import json
import sys
import inspect
# read these for more info:
# https://chat.openai.com/share/9341df1e-65c0-4fd8-87dc-80e0dc9fa5bc
# https://chat.openai.com/share/193482c6-e7b3-4022-b42b-cd2530efb839
dotenv.load_dotenv()
openai.api_key = os.getenv("OPENAI_API_KEY")
async def run_chat_prompt(prompt, model="gpt-4"):
completion = openai.ChatCompletion.create(
model=model,
messages=[
{"role": "user", "content": prompt}
]
)
return (completion.choices[0].message.content)
def wrap_error(e: Exception, description: str = None):
exc_type, exc_value, exc_traceback = sys.exc_info()
filename = exc_traceback.tb_frame.f_code.co_filename
line_number = exc_traceback.tb_lineno
func_name = exc_traceback.tb_frame.f_code.co_name
module = inspect.getmodule(exc_traceback.tb_frame)
function_obj = getattr(module, func_name)
# Get and print the source code of the function
func_code = inspect.getsource(function_obj)
error_message = f"This error message occurred because of '{str(e)}' at line {line_number} in file {filename}, function {func_name}. The exception type is {exc_type}. The function is: \n\n```python\n{func_code}\n```\n\n How do I fix this?"
if description:
error_message += f'Other details: {description}'
return error_message
import error_wrap
async def main():
try:
raise Exception("test")
except Exception as e:
potential_fix = await error_wrap.run_chat_prompt(error_wrap.wrap_error(e))
# an optional description can be added with something along the lines of: error_wrap.wrap_error(e, "here's some specific info passed in by me, a human")
return {"error": str(e), "potential_fix": potential_fix}
@Cytranics
Copy link

Cytranics commented Jun 7, 2023

Great idea man, thank you. I took it a step further and make it a decorator, so now all I do is @gpt_error and decorate all my functions and it prints the responses to the console.

Thanks for the idea my friend.

Here is my contribution thanks to your code. It handles async or non async functions and now supports azure and openai. I should prob wrap the openai call in some error handling, but....You all can do that ;)

import asyncio
import openai
import sys
import traceback
import inspect
from functools import wraps
import termcolor

openai.api_type = "azure"
openai.api_base = os.environ.get("AZURE_URL")
openai.api_version = os.environ.get("API_VERSION")
openai.api_key = os.environ.get("AZURE_API_KEY")
engine = os.environ.get("AZURE_MODEL_NAME")

def gpt_error(func):
    if inspect.iscoroutinefunction(func):
        @wraps(func)
        async def async_wrapper(*args, **kwargs):
            try:
                return await func(*args, **kwargs)
            except Exception as e:
                return await handle_error(e, func)
        return async_wrapper
    else:
        @wraps(func)
        def sync_wrapper(*args, **kwargs):
            try:
                return func(*args, **kwargs)
            except Exception as e:
                print(termcolor.colored(f"Error: {e}", color="red"))
                error_message = asyncio.run(handle_error(e, func))
                print(error_message)
                return error_message
        return sync_wrapper

async def handle_error(e, func):
    exc_type, exc_value, exc_traceback = sys.exc_info()
    stack_trace = traceback.extract_tb(exc_traceback)
    last_trace = stack_trace[-1]
    filename = last_trace[0]
    line_number = last_trace[1]
    func_name = last_trace[2]

    # Get and print the source code of the function
    func_code = inspect.getsource(func)

    error_message = f"This error message occurred because of '{str(e)}' at line {line_number} in file {filename}, function {func_name}. The exception type is {exc_type}. The function is: \n\n```python\n{func_code}\n```\n\n Provide a detailed explanation on why this error occurred and sample python code on how to rectify the problem."

    # Make an API call to OpenAI chat with the error message
    completion = await openai.ChatCompletion.acreate(
        engine="chatGPT",
        messages=[
            {"role": "user", "content": error_message}
        ]
    )

    
    return completion.choices[0].message.content

@bramses
Copy link
Author

bramses commented Jun 7, 2023

I took it a step further and make it a decorator, so now all I do is @gpt_error and decorate all my functions and it prints the responses to the console.

@Cytranics that's an amazing idea! I didn't think of using decorators but that is definitely the way to do it. Thank you for the improvement! I'm decorating my codebase with it as we speak!! :)

@CarolusIII
Copy link

Hello Sir,
your detailed Readme of the Bramses' Highly Opinionated Vault 2023 got my attention.
I would love to learn more on your thoughts and ideas.
It is probably due to me being not that experienced in the use of github, but I really could not find any way to download a zipped version of your vault that I could use.
Hopefully you can help, but anyway I wish you a lot of success with your adventures.
I subscribed to your newsletter.
All the best,
Wim Claessen
Netherlands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment