slintak/_README.md

## _README.md

      
    Raw
  

              _README.md
            
          
    This gist contains a proof of concept project (hacked together during one evening) aimed at
connecting an AI chat assistant to the Meshtastic network, enhancing its capabilities.
The assistant is designed to provide concise and helpful responses within the constraints
of the Meshtastic environment.
Project Overview:

This project brings together two favorite elements: the Meshtastic network and an AI chat assistant.
The assistant is based on the "FuseChat-7B-VaRM-Q5_K_M" model, and the code is written in Python.
The code is optimized to run on an ordinary PC without a GPU. I tested it on my old i5-2400 CPU
with 16 GB of RAM. In such cases, responses may take around 30-60 seconds.
Functionality:

The assistant receives packet payload, Received Signal Strength Indication (RSSI), and
Signal-to-Noise Ratio (SNR) from the Meshtastic network. It generates responses with a maximum length
of 237 characters, aiming to be concise and helpful.
Getting Started:

To run the project, ensure you have Python installed along with the AI model and required dependencies
specified in requirements.txt. Additionally, make sure to have Meshtastic devices set up and connected
to the USB or to the network.
Usage:


Clone this repository to your local machine.
Install dependencies using pip install -r requirements.txt.
Download model huggingface-cli download LoneStriker/FuseChat-7B-VaRM-GGUF FuseChat-7B-VaRM-Q5_K_M.gguf --local-dir ./models --local-dir-use-symlinks False
Run the Python script main.py.

Notes:

This project is just a proof of concept and may require further tweaking for optimal performance.
Any feedback is welcome;
feel free to reach out.
Enjoy exploring Meshtastic with AI assistance!
If you enjoy this project, be sure to check out my other creation, MakerDeals.eu. This website
simplifies the process of finding, filtering, and comparing 3D printing filaments available on Amazon.

  
## main.py
import time
from meshtastic.tcp_interface import TCPInterface
from meshtastic.serial_interface import SerialInterface
from llama_cpp import Llama
from pubsub import pub


PROMPT = '''
    You are an assistant working on the Meshtastic LoRa network.
    You can respond in English or Czech languages.
    Your responses are straight to the point, short, helpful and mostly one sentence.
    The maximum length of your response is 237 characters.
    Input from the user contains signal quality information as a JSON object.
    These are for context and should not be repeated in the response!
'''


llm = Llama(
    model_path="./models/FuseChat-7B-VaRM-Q5_K_M.gguf",
    n_threads=4,
    use_mmap=True,
    use_mlock=True,
    n_ctx=4048,
)


def onReceive(packet, interface):
    data = packet.get('decoded')
    if data.get('portnum') == 'TEXT_MESSAGE_APP':
        payload = data.get('payload', b'').decode('utf-8')
        rssi = packet['rxRssi'] or '?'
        snr = packet['rxSnr'] or '?'
        content = f'{payload} {{"rssi":"{rssi} dBm","snr":"{snr} dB"}}'
        output = llm.create_chat_completion(
            messages=[
                {"role": "system", "content": PROMPT},
                {"role": "user", "content": content},
            ]
        )
        response = output["choices"][0]["message"]["content"]

        print(f"Received: {content}")
        print(f"Response: {response}\n")
        interface.sendText(response)


def onConnection(interface, topic=pub.AUTO_TOPIC):
    # interface.sendText("AI assistant is alive!")
    pass


pub.subscribe(onReceive, "meshtastic.receive")
pub.subscribe(onConnection, "meshtastic.connection.established")
interface = SerialInterface(devPath='/dev/ttyACM0')
# interface = TCPInterface(hostname='192.168.0.2', connectNow=True)

while True:
    print('Idle.')
    time.sleep(60)

interface.close()

## requirements.txt
huggingface-hub>=0.17.1
llama-cpp-python
meshtastic
	import time
	from meshtastic.tcp_interface import TCPInterface
	from meshtastic.serial_interface import SerialInterface
	from llama_cpp import Llama
	from pubsub import pub


	PROMPT = '''
	You are an assistant working on the Meshtastic LoRa network.
	You can respond in English or Czech languages.
	Your responses are straight to the point, short, helpful and mostly one sentence.
	The maximum length of your response is 237 characters.
	Input from the user contains signal quality information as a JSON object.
	These are for context and should not be repeated in the response!
	'''


	llm = Llama(
	model_path="./models/FuseChat-7B-VaRM-Q5_K_M.gguf",
	n_threads=4,
	use_mmap=True,
	use_mlock=True,
	n_ctx=4048,
	)


	def onReceive(packet, interface):
	data = packet.get('decoded')
	if data.get('portnum') == 'TEXT_MESSAGE_APP':
	payload = data.get('payload', b'').decode('utf-8')
	rssi = packet['rxRssi'] or '?'
	snr = packet['rxSnr'] or '?'
	content = f'{payload} {{"rssi":"{rssi} dBm","snr":"{snr} dB"}}'
	output = llm.create_chat_completion(
	messages=[
	{"role": "system", "content": PROMPT},
	{"role": "user", "content": content},
	]
	)
	response = output["choices"][0]["message"]["content"]

	print(f"Received: {content}")
	print(f"Response: {response}\n")
	interface.sendText(response)


	def onConnection(interface, topic=pub.AUTO_TOPIC):
	# interface.sendText("AI assistant is alive!")
	pass


	pub.subscribe(onReceive, "meshtastic.receive")
	pub.subscribe(onConnection, "meshtastic.connection.established")
	interface = SerialInterface(devPath='/dev/ttyACM0')
	# interface = TCPInterface(hostname='192.168.0.2', connectNow=True)

	while True:
	print('Idle.')
	time.sleep(60)

	interface.close()