MemGPT + Open-Source Models

gistfile1.txt

# spin up a runpod GPU with TextGen WebUI, no need to use openai flag
# you can also use TextGen WebUI on your local computer
git clone https://github.com/cpacker/MemGPT.git
cd MemGPT
export OPENAI_API_BASE=https://some-stuff-5000.proxy.runpod.net # using runpod
# export OPENAI_API_BASE=localhost:5000 # using local computer
BACKEND_TYPE=webui
python3 main.py --no_verify

its "export BACKEND_TYPE=webui"

Windows - Set OPEN_API_BASE=Http://localhost:5000

I got an error api error although at http://localhost:7860/ my textgen webui is running. in session tab my api is enabled. Model i am using: mistralai_Mistral-7B-v0.1

Warning: no wrapper specified for local LLM, using the default wrapper
step() failed
user_message = None
error = API call got non-200 response code for address: http://localhost:5000/api/v1/generate. Make sure that the web UI server is running and reachable
at http://localhost:5000/api/v1/generate.
API call got non-200 response code for address: http://localhost:5000/api/v1/generate. Make sure that the web UI server is running and reachable at
http://localhost:5000/api/v1/generate.
An exception ocurred when running agent.step():
Traceback (most recent call last):
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/main.py", line 524, in run_agent_loop
) = await memgpt_agent.step(user_message, first_message=False, skip_verify=no_verify)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/agent.py", line 1013, in step
raise e
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/agent.py", line 958, in step
response = await get_ai_reply_async(model=self.model, message_sequence=input_message_sequence, functions=self.functions)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/agent.py", line 159, in get_ai_reply_async
raise e
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/agent.py", line 140, in get_ai_reply_async
response = await acreate(
^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/openai_tools.py", line 102, in wrapper
raise e
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/openai_tools.py", line 82, in wrapper
return await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/openai_tools.py", line 111, in acompletions_with_backoff
return await get_chat_completion(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/local_llm/chat_completion_proxy.py", line 50, in get_chat_completion
result = get_webui_completion(prompt)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/local_llm/webui/api.py", line 32, in get_webui_completion
raise Exception(
Exception: API call got non-200 response code for address: http://localhost:5000/api/v1/generate. Make sure that the web UI server is running and reachable at http://localhost:5000/api/v1/generate.

For LM Studio try this:
export OPENAI_API_BASE=http://localhost:1234
export BACKEND_TYPE=lmstudio

hi @mschwarzmueller , i used llmstudio. but after the first response from llmstudio "Hello, how can I assist you today?". I got following error, can please guid me.

user_message = None
error = Failed to decode JSON from LLM output:
{ "Hello, how can I assist you today?"
Failed to decode JSON from LLM output:
{ "Hello, how can I assist you today?"
An exception ocurred when running agent.step():
Traceback (most recent call last):
File "/Users/bhagatsingh/vscode/ai/MemGPT/memgpt/local_llm/llm_chat_completion_wrappers/airoboros.py", line 392, in output_to_chat_completion_response
function_json_output = json.loads(raw_llm_output)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/anaconda3/envs/memgpt/lib/python3.11/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/anaconda3/envs/memgpt/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/bhagatsingh/anaconda3/envs/memgpt/lib/python3.11/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 39 (char 38)

During handling of the above exception, another exception occurred:

@bhagat9198 try to use the model: dolphin 2 1 mistral 7b q6_k

In the settings.py file (in lmstudio folder):

SIMPLE = {
    "stop": [
        "\nUSER:",
        "\nASSISTANT:",
        "\nFUNCTION RETURN:",
        # '\n' +
        # '</s>',
        # '<|',
        # '\n#',
        # '\n\n\n',
    ],
    "max_tokens": 1000,
}

I got some results with this but a lot of bugs too.

Hi @mvnigro thanks alot.

MEMGPT was working fine, now getting this error

ModuleNotFoundError: No module named 'memgpt.local_llm.chat_completion_proxy'

In windows command line i get the error "Directory 'static/' does not exist "

python3 main.py
Traceback (most recent call last):
File "C:\Users\uname\MemGPT\MemGPT\main.py", line 1, in
from memgpt.main import app
File "C:\Users\uname\MemGPT\MemGPT\memgpt\main.py", line 17, in
from .interface import print_messages
File "C:\Users\uname\MemGPT\MemGPT\memgpt\interface.py", line 6, in
from memgpt.utils import printd
File "C:\Users\uname\MemGPT\MemGPT\memgpt\utils.py", line 12, in
import fitz
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\fitz_init_.py", line 1, in
from frontend import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend_init_.py", line 1, in
from .events import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events_init_.py", line 1, in
from .clipboard import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events\clipboard.py", line 2, in
from ..dom import Event
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dom.py", line 439, in
from . import dispatcher
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dispatcher.py", line 15, in
from . import config, server
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-pTraceback (most recent call last):
File "C:\Users\uname\MemGPT\MemGPT\main.py", line 1, in
from memgpt.main import app
File "C:\Users\uname\MemGPT\MemGPT\memgpt\main.py", line 17, in
from .interface import print_messages
File "C:\Users\uname\MemGPT\MemGPT\memgpt\interface.py", line 6, in
from memgpt.utils import printd
File "C:\Users\uname\MemGPT\MemGPT\memgpt\utils.py", line 12, in
import fitz
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\fitz_init_.py", line 1, in
from frontend import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend_init_.py", line 1, in
from .events import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events_init_.py", line 1, in
from .clipboard import *
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events\clipboard.py", line 2, in
from ..dom import Event
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dom.py", line 439, in
from . import dispatcher
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dispatcher.py", line 15, in
from . import config, server
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\server.py", line 24, in
app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME)
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\starlette\staticfiles.py", line 57, in init
raise RuntimeError(f"Directory '{directory}' does not exist")
RuntimeError: Directory 'static/' does not existackages\frontend\server.py", line 24, in
app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME)
File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\starlette\staticfiles.py", line 57, in init
raise RuntimeError(f"Directory '{directory}' does not exist")
RuntimeError: Directory 'static/' does not exist

In windows command line i get the error "Directory 'static/' does not exist "

python3 main.py Traceback (most recent call last): File "C:\Users\uname\MemGPT\MemGPT\main.py", line 1, in from memgpt.main import app File "C:\Users\uname\MemGPT\MemGPT\memgpt\main.py", line 17, in from .interface import print_messages File "C:\Users\uname\MemGPT\MemGPT\memgpt\interface.py", line 6, in from memgpt.utils import printd File "C:\Users\uname\MemGPT\MemGPT\memgpt\utils.py", line 12, in import fitz File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\fitz__init__.py", line 1, in from frontend import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend__init__.py", line 1, in from .events import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events__init__.py", line 1, in from .clipboard import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events\clipboard.py", line 2, in from ..dom import Event File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dom.py", line 439, in from . import dispatcher File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dispatcher.py", line 15, in from . import config, server File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-pTraceback (most recent call last): File "C:\Users\uname\MemGPT\MemGPT\main.py", line 1, in from memgpt.main import app File "C:\Users\uname\MemGPT\MemGPT\memgpt\main.py", line 17, in from .interface import print_messages File "C:\Users\uname\MemGPT\MemGPT\memgpt\interface.py", line 6, in from memgpt.utils import printd File "C:\Users\uname\MemGPT\MemGPT\memgpt\utils.py", line 12, in import fitz File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\fitz__init__.py", line 1, in from frontend import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend__init__.py", line 1, in from .events import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events__init__.py", line 1, in from .clipboard import * File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\events\clipboard.py", line 2, in from ..dom import Event File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dom.py", line 439, in from . import dispatcher File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\dispatcher.py", line 15, in from . import config, server File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\frontend\server.py", line 24, in app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME) File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\starlette\staticfiles.py", line 57, in init raise RuntimeError(f"Directory '{directory}' does not exist") RuntimeError: Directory 'static/' does not existackages\frontend\server.py", line 24, in app.mount(config.STATIC_ROUTE, StaticFiles(directory=config.STATIC_DIRECTORY), name=config.STATIC_NAME) File "C:\Users\uname\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\starlette\staticfiles.py", line 57, in init raise RuntimeError(f"Directory '{directory}' does not exist") RuntimeError: Directory 'static/' does not exist

same error on mac