Skip to content

Instantly share code, notes, and snippets.

@indiejoseph

indiejoseph/vision-of-ai-smart-watch.md

Last active May 5, 2024 09:53
Show Gist options
  • Save indiejoseph/cc82398d14b020a4eba1f95ed7743b37 to your computer and use it in GitHub Desktop.
Save indiejoseph/cc82398d14b020a4eba1f95ed7743b37 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
vision-of-ai-smart-watch.md

Exploring the Vision of AI Smart Watch

Here is some thoughts of mine on the vision of AI Smart Watch. I hope you enjoy it.

Introduction

Running LLM(Large language model) on a embedding device is a trendy topic in the field of AI. The AI Smart Watch is a typical example of such devices. It is also a challenging task because of the limited resources of the device. In this article, I will explore the vision and the solution of AI Smart Watch.

Vision

When we talk about LLM, most people will think of ChatGPT liked Chatbot or a personal assistant like Siri. However, the user experience of these applications on a smartwatch is not good. The screen of the smartwatch is too small to display the long text generated by the LLM. The input method of the smartwatch is also not good for typing long text. So, we also have to consider the user experience when we design the LLM on a smartwatch.

Large language models can be used in many other scenarios. For example, we can use it to generate the summary of the news, the answer of the question, the translation of the text, etc. These applications are more suitable for the smartwatch. The user can get the information they want quickly and easily.

For example, Voice assistant is a good application for the smartwatch. The user can ask the voice assistant to do some tasks, such as setting the alarm, sending the message, etc. The voice assistant can generate the text based on the user's voice input and give the user the answer. This application can also be used by other applications on the smartwatch, such as the health monitoring application, the transaction log analysis application, etc. and this case we might also need to have deploy Automatic Speech Recognition(ASR) model and Text-to-Speech(TTS) model on the smartwatch to make the voice assistant work. This would also be a challenging task because of the limited resources of the device.

To me we are kind of adding a Siri liked assistant to the smartwatch, but with more capabilities and possibilites for developers to build on top of it.

Use Cases

Large language models is a general-purpose model, it can be used in many scenarios. harnessing its in-context learning capabilities, we can build a lot of applications on the smartwatch. Here are some examples:

  1. Transaction log analysis: The LLM can analyze the blockchain transaction log of the user and give the user some insights about their transaction status.
  2. Health monitoring: The LLM can monitor the health status of the user and give the user some suggestions about their health.
  3. Personal assistant: The LLM can help the user to manage their schedule, remind them of the important events, etc.
  4. Language translation: The LLM can translate the text in the foreign language to the user's language.
  5. News summary: The LLM can generate the summary of the news for the user.
  6. Crypto price monitoring: The LLM can monitor the price of the cryptocurrency and give the user some insights about the price trend.

Here are some examples of the use cases of the AI Smart Watch. I hope you can get some inspiration from them.

LLM as a Service

LLM as a Service is a new concept in the field of AI. In the context of the smartwatch, application developers can use the LLM as a Service to build their applications. The LLM as a Service can provide the API for the developers to interact with the LLM. The developers can use the API to generate the text, answer the question, etc. The LLM pre-trained model can be chosen by the user to fit their language preference, and the developer can use it to build their application.

Solution

It is a challenging task to run LLM on a smartwatch because of the limited resources of the device, but it is necessary for the user experience and data privacy. Here are some solutions to this problem:

  1. Open source community has released various lightweight models, such as Qwen1.5, Gemma, Phi-3, etc. The size small from 0.5B to 3B, which is suitable for the smartwatch.

  2. Model quantization: We can quantize the model to reduce the size of the model. This will make the model run faster on the smartwatch, but the downside is that the accuracy of the model will be reduced. A model can be quantized to 8-bit or even lower, which can reduce the size of the model by 4 times.

  3. Hybrid solution: We can use a hybrid solution to run the model on the smartwatch. We can run the model on the cloud and use the smartwatch as a thin client to interact with the model. This will reduce the resource consumption of the smartwatch and improve the user experience. We should also consider the data privacy issue when we use this solution.

Depends on the use case, we can choose the suitable solution to run the LLM on the smartwatch.

Data Privacy

Data privacy is a big concern when we run LLM on the smartwatch. The user's data is very sensitive and should be protected. We should consider the following aspects when we design the AI Smart Watch:

  1. Data encryption: The user's data should be encrypted when it is stored on the smartwatch. This will protect the user's data from being stolen by the attacker.

  2. Data anonymization: The user's data should be anonymized when it is sent to the cloud for processing. This will protect the user's privacy and prevent the attacker from tracking the user's data.

  3. Data deletion: The user's data should be deleted after it is processed. This will protect the user's privacy and prevent the attacker from stealing the user's data.

  4. Data access control: The user's data should be accessed by the authorized person only. This will protect the user's privacy and prevent the attacker from stealing the user's data. the LLM should not be able to access internet without user's permission. we could get reference of Android's permission system.

We should consider the data privacy issue when we design the AI Smart Watch. The user's data is very sensitive and should be protected.

Safety

Open source models are not always safe, we should consider the safety issue when we run the LLM on the smartwatch. The model should be tested and verified before it is deployed on the smartwatch. Here are some aspects we should consider when deploying the model on the smartwatch:

  1. Harassment

  2. Hate speech

  3. Sexually explicity

  4. Dangerous

Solution for Safety

There are some solutions to ensure the safety of the model:

  1. Preference Optimization: The secret sauce of ChatGPT success is the model undergone the RLHF(Reinforcement Learning from Human Feedback) to optimize the model to be more human-like and less harmful. there are some other ways also could achieve this, like DPO(Direct Preference Optimization) and ORPO(Monolithic Preference Optimization), we could use these methods to optimize the model to be more safe.

  2. Safety Detection: Google's Gemini Pro is a good example of safety detection, it could detect the harmful content and prevent it from being generated, along of the LLM generation process, the safety detection model could be used to detect the harmful content and prevent it from being generated.

Beside of the above solutions, we should also provide a way for the user to report the harmful content, and we should take action to remove the harmful content from the model.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment