Creating a Quick Action for Sending Selected Text to Large Language Models on macOS

README.md

Introduction

This instruction explains how to create a Quick Action in macOS that can send selected text directly to your preferred large language model.

The workflow is inspired by this Hacker News post of eliya_confiant: https://news.ycombinator.com/item?id=39592297.

Prerequisites

To follow this guide, you will need the following tools:

1. glow: for rendering markdown from CLI
2. Ollama (optional): a local LLM interface
3. llm: for CLI integration (Optional)

Run the following command to install them:

brew install glow ollama
brew install llm  # optional

Configuring the LLM

Pull a desired language model from Ollama. I recommend trying out different models for personal use. For example, use llama3 for general-purpose tasks or qwen2:0.5 for instant responses.

Configure llm (Optional)

Using OpenAI or gpt4all

If you have an OpenAI API key or if you prefer using gpt4all, follow the instructions on llm's README.

Using Ollama as your interface

1. Pull the model files, for example: ollama pull mistral
2. Create extra-openai-models.yaml under $HOME/Library/Application Support/io.datasette.llm.
3. Add the following settings to the file:

- model_id: mistral
  model_name: mistral
  api_base: "http://localhost:11434/v1/"

4. You can add as many models as you want. Simply change the model_id and model_name.

Configuring for Automation

Creating a Quick Action

1. Open Automator, create a new Quick Action.
2. Find Run Shell Script from the sidebar and drag it to the configuration panel.
3. Copy the script from llm-agent.sh, and paste it into the Run Shell Script action's text area.
4. Save. Now you can select text from any application, right-click, and choose your new Quick Action to send the text to your preferred LLM.

Configuring a Keyboard Shortcut

1. Open System Settings.
2. Go to Keyboard and open Keyboard Shortcuts....
3. From Services > Text, find the Quick Action you just created.
4. Set your desired keyboard combination, and press Done.

Notes

The result is automatically copied to your clipboard. If you want to see the result in rendered markdown, remember to keep your terminal app open.

llm-agent.sh

#!/bin/zsh

# Path to the LLM executable (replace with actual path if different)
# Option 1: Use ollama (https://ollama.com/)
LLM_COMMAND="/usr/local/bin/ollama"
LLM_ATTR="run"

# Option 2: Use llm to connect to various LLM backends (https://github.com/simonw/llm)
# Modify $HOME/Library/Application Support/io.datasette.llm to extend available LLMs.
# LLM_COMMAND="/opt/homebrew/bin/llm"
# LLM_ATTR="-m"

# Name of the LLM model to use (change to desired model name)
LLM_MODEL="llama3"

# Define available agents and their prompts
declare -A agentMap
agentMap["Passthrough"]=" "
agentMap["EditingAssistant"]="Follow my instruction and respond in markdown format. Revise the following text for clarity, grammar, word choice, and sentence structure. Maintain a neutral tone and conversational style.\nText:"
agentMap["ExplainCode"]="Follow my instruction and respond in markdown format. Break down the following code line by line. Explain what each line does and how it contributes to the overall logic of the code. If possible, describe any potential business applications of this code. Try reasoning the code step by step.\nCode:"
agentMap["Summary"]="Follow my instruction and respond in markdown format. Summarize the following in a short and clear way. Be sure to include all the main points.\nText:"
## Add more agents here
# agentMap["AgentName"]="Your Prompt"

# Build list of agents from the map (to display in the AppleScript prompt)
agents=""
for key in ${(k)agentMap}
do
    if [[ $agents == "" ]]; then
        agents="$key"
        continue
    fi
    agents="$agents, $key"
done

# Prompt user to choose an agent using osascript
default_agent="EditingAssistant"
selected_agent=$(osascript -e "choose from list {$agents} with prompt \"Select an agent to use:\" default items {\"$default_agent\"}")

if [ "$selected_agent" = "false" ]; then
    exit
fi

# Build the prompt based on chosen agent and escaped arguments
escaped_args=""
for arg in "$@"; do
    escaped_arg=$(printf '%s\n' "$arg" | sed "s/'/'\\\\''/g")
    escaped_args="$escaped_args '$escaped_arg'"
done
prompt="${agentMap[\"$selected_agent\"]} $escaped_args"

# Run LLM command, escape result for safety
result=$($LLM_COMMAND $LLM_ATTR $LLM_MODEL $prompt)
escapedResult=$(echo "$result" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | awk '{printf "%s\\n", $0}' ORS='')

# Copy result to clipboard and display notification
osascript -e "set the clipboard to \"$escapedResult\""
osascript -e "display notification \"LLM results copied to clipboard\""

# Open iTerm window with temporary file for displaying result
temp_file=$(mktemp)
echo "$escapedResult" > "$temp_file"

# Option 1: Display results in Terminal
osascriptCmd="tell application \"Terminal\"
    do script \"glow -p ${temp_file} && exit\"
end tell"
osascript -e "$osascriptCmd"

# Option 2: Display results in iTerm2
# osascriptCmd="tell application \"iTerm2\"
#     create window with default profile
#     tell current session of current window
#         write text \"glow -p ${temp_file} && exit\"
#     end tell
# end tell"