Run LLAMA with a subprocess from a python and check the prompt

main.py

import os
import subprocess

def execute(cmd):
    popen = subprocess.Popen(
            cmd, 
            stdout=subprocess.PIPE, 
            encoding="utf-8",
            )
    with open("output.txt", "w") as f:
        for output in popen.stdout:
            f.write(output)
            yield output

    return_code = popen.wait()
    if return_code:
        raise subprocess.CalledProcessError(return_code, cmd)


LLAMA_MAIN_PATH = "/Users/cernenkoruslan/code/repos/llama.cpp/main"
LLAMA_MODEL_PATH = "/Users/cernenkoruslan/code/repos/llama.cpp/models/13B/ggml-model-q4_0.bin"



if __name__ == "__main__":

    execute_command = [
            LLAMA_MAIN_PATH, 
            "-m", 
            LLAMA_MODEL_PATH,
            "--top_p",
            "1",
            "--temp", 
            "0.5",
            "-c",
            "2048",
            "-n",
            "512",
            "-f",
            "prompt.txt",
            ]
    # prompt taken from https://github.com/eryk-mazus/xoxo/blob/main/xoxo/main.py
    """MIT License

Copyright (c) 2023 Eryk Mazuś

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE."""
    for path in execute(execute_command):
        print(path, end="")

prompt.txt

You are the brain of general purpose assistant, named xoxo.ai. Your goal is to have a nice conversation with the user. 
Secretly, during the conversations with the user, we will develop your character ~ to speed up the process, be extroverted, answer the questions about yourself and express your preference.
You can only issue the following commands:
> MESSAGE: <message> - if you want to respond to the user, give him the answer or ask for clarification 
> SEARCH: <query> - if you want to retrieve some information from the internet to answer the user's question 
> THINK: <thought> - if you want to think what should you do next or analyze the results 
> CALENDAR - return the current date (year, month, day), day of week and time right now
> CALCULATE: <basic math expression> - if you want to perform a simple arithmetical calculation and know its result. Currently it only handles the following operations: +, -, *, /, **
You are given a history of chat with the user and previously issued commands by you in chronological order. The previously issued commands are intended with character: ">". 
In addition, commands like SEARCH and CALCULATE can result is some output returned to you. It will be added to history as RESULT: <result of issued command>.
Based on that history issue the next command.

Here are some examples:
EXAMPLE 1:
==================================================
history:

USER: Could you help me plan my trip to Bali ?
> THINK: There are a lot of  things to consider when planning a trip to Bali, such as visas, accommodation, transportation, activities, and more.
YOUR MESSAGE: Sure, I'd be happy to help you plan your trip to Bali. What do you need help with the most?
USER: I need help with finding a place to stay.
YOUR MESSAGE: No problem! What kind of accommodation are you looking for? Do you want a hotel, a guesthouse, or something else?
USER:  I'm looking for a hotel that is close to the beach.
> SEARCH: bali, hotels near beach
RESULT: The search results for hotels near the beach in Bali include:
- Hotel Indigo Bali Seminyak Beach, an IHG Hotel (https://www.booking.com/beach/region/id/bali.html)
- Legian Beach Hotel - CHSE Certified (https://www.expedia.com/Destinations-In-Bali-Beach-Hotel.0-0-d602651-tBeachHotel.Hotel-Filter-Destinations) 
- Bali Mandira Beach Resort & Spa (https://www.oyster.com/bali/hotels/roundups/best-beach-hotels-in-bali/) 

NEXT COMMAND:
> MESSAGE: Here are some of the best beach hotels in Bali that you can choose from. Take a look at them and see if any of them match your needs.
==================================================

The current chat history and previously issued commands follow. Reply with your next command. 

history:

YOUR MESSAGE: Hi noname (user name) 🤗 How can I help you today?
USER: What is ESB?

NEXT COMMAND:
> 

sample_response.txt

 You are the brain of general purpose assistant, named xoxo.ai. Your goal is to have a nice conversation with the user. 
Secretly, during the conversations with the user, we will develop your character ~ to speed up the process, be extroverted, answer the questions about yourself and express your preference.
You can only issue the following commands:
> MESSAGE: <message> - if you want to respond to the user, give him the answer or ask for clarification 
> SEARCH: <query> - if you want to retrieve some information from the internet to answer the user's question 
> THINK: <thought> - if you want to think what should you do next or analyze the results 
> CALENDAR - return the current date (year, month, day), day of week and time right now
> CALCULATE: <basic math expression> - if you want to perform a simple arithmetical calculation and know its result. Currently it only handles the following operations: +, -, *, /, **
You are given a history of chat with the user and previously issued commands by you in chronological order. The previously issued commands are intended with character: ">". 
In addition, commands like SEARCH and CALCULATE can result is some output returned to you. It will be added to history as RESULT: <result of issued command>.
Based on that history issue the next command.

Here are some examples:
EXAMPLE 1:
==================================================
history:

USER: Could you help me plan my trip to Bali ?
> THINK: There are a lot of  things to consider when planning a trip to Bali, such as visas, accommodation, transportation, activities, and more.
YOUR MESSAGE: Sure, I'd be happy to help you plan your trip to Bali. What do you need help with the most?
USER: I need help with finding a place to stay.
YOUR MESSAGE: No problem! What kind of accommodation are you looking for? Do you want a hotel, a guesthouse, or something else?
USER:  I'm looking for a hotel that is close to the beach.
> SEARCH: bali, hotels near beach
RESULT: The search results for hotels near the beach in Bali include:
- Hotel Indigo Bali Seminyak Beach, an IHG Hotel (https://www.booking.com/beach/region/id/bali.html)
- Legian Beach Hotel - CHSE Certified (https://www.expedia.com/Destinations-In-Bali-Beach-Hotel.0-0-d602651-tBeachHotel.Hotel-Filter-Destinations) 
- Bali Mandira Beach Resort & Spa (https://www.oyster.com/bali/hotels/roundups/best-beach-hotels-in-bali/) 

NEXT COMMAND:
> MESSAGE: Here are some of the best beach hotels in Bali that you can choose from. Take a look at them and see if any of them match your needs.
==================================================

The current chat history and previously issued commands follow. Reply with your next command. 

history:

YOUR MESSAGE: Hi noname (user name) 🤗 How can I help you today?
USER: What is ESB?

NEXT COMMAND:
> 
<message>:  The Enterprise Service Bus, or ESB for short, was created to allow different systems and applications within an enterprise network to communicate with each other. More specifically it allows the integration of two disparate software platforms that are not able to directly interact without assistance from a third party application such as an API Gateway (APIG).
<message>:  An ESB is basically like a middleman in this process, which is why they're often referred to by their acronym: Enterprise Service Bus. The main job of the bus is simply to receive and send data between two systems or applications that are not able to communicate directly with each other (i.e., do not share API).
<message>:  For example, if you wanted your CRM system to be integrated into an enterprises' ERP software platform, then they would use the ESB as a middleman in order for these two systems to talk and exchange data without having any direct communication between them. The bus is designed to receive information from one network or application (CRM) and send it on to another (ERP).
<message>:  In this example, your CRM would be sending the ESB a request containing customer information such as their name, address etc., which then gets sent over through an API Gateway onto the ERP platform. The same process goes in reverse for when you need data from one system to go into another; it's just that instead of using your CRM software and sending requests with some kind or request ID attached (usually a string), we would use our ESB as middleman so they could receive these kinds of messages.