Connect to Ollama with JS

ollama.md

Contents:

learning about Ollama

fat_lipped_llama

learning about Ollama

Ollama works like a normal REST API

There are two sides:

1. The Client
   ollama run llama2
   the REPL: read, execute, print, loop
   CLI is an API client
   

2. The Service
   ollama serve
   not interactive
   runs as a service
   publishes the api
   

Github documentation avalaible at https://github.com/ollama/ollama/blob/main/docs/api.md

There are two primary endpoints, api/chat and api/generate

use chat for conversations, where you need memory. Does require extra work if not necessary.
localhost:11434/api/chat

use generate for one off conversations, no memory.
localhost:11434/api/generate

Request example:

{
  'model': 'llama2',
  'prompt': 'How do you feel about cheese?',
  'images': ['iVBOR...'  Base 64 encoded
}

Images must be base 64 encoded. Images is an array, but as of now only the first image is recognized

Response example:

{
  'model': 'llava',
  'created_at': '2024-06-12-TIMESTAMP',
  'response': 'image',
  'done': false
}

Responses are a series/stream of JSON tokens. Most of them are full of words, but some are just parts of words or other tokens. The last response in the stream will have 'done' set to true and provides 'context' that you can pass on to the next call if you're having a chat conversation.

In our initial request we can set 'stream' to false if we'd prefer to have all the responses joined together. However we'll have to wait for the full stream to be generated before we get any response, this could be slow.

Turning streaming off might be helpful if you're returning a full JSON object. If we send the optional key 'format': 'json' in the request we'll receive a response as a single JSON object. Ideally, also mention this in the prompt and give an example of the schema you'd like.

'options' 'system' and 'template' in the request override what is set in the model file. for example 'system': 'The user will provide a concept. Explain the concept in an easy to understand manner that even a 6 year old could understand.'

---

fat_lipped_llama

Ideas for a front end. Fat lipped llama. Its a madlibs style generator that modifies the 'options' or system or template, I guess whichever field works. Then the user would still provide the prompt, but hopefully the output would be funny... Can we get it to read the response? Even better!

Lets modify this base sentence, 6 of the words will be choosable? or random?:

const options = `The ${identifier} will ${provideVerb} a ${subjectMatter}. ${ExplainVerb} the ${subjectMatter} in a ${difficultyVerb} to ${understandVerb} manner that even a ${age} year old ${identifier} could ${understandVerb}.

What's this roughly going to look like? Sudo code incomming:

randomWord(words) => {
  // const randomIndex = Math( something...   x10s or whatevers
  console.log(' num between 0-9?? :', randomIndex);
  return words[randomIndex];
};

const identifier = randomWord(identifiers);
const provideVerb = randomWord(provideVerbs);
const subjectMatter = randomWord(subjectMatters);
const ExplainVerb = randomWord(explainVerbs);
const difficultyVerb = randomWord(difficultyVerbs);
const understandVerb = randomWord(understandVerbs);
const age = randomWord(ages);

const identifiers = [
  "Pirate",
  "Ninja",
  "Chef",
  "Scientist",
  "Teacher",
  "Artist",
  "Astronaut",
  "Detective",
  "Engineer",
  "Explorer"
];

const provideVerbs = [
  "shoot",
  "hurl",
  "toss",
  "chuck",
  "sling",
  "plop",
  "hand over",
  "drop kick",
  "catapult",
  "yeet"
];

const subjectMatters = [
  "fart",
  "perfume",
  "pickle",
  "cucumber",
  "unicorn",
  "donkey",
  "spaghetti",
  "empty plate",
  "sneeze",
  "silence"
];

const explainVerbs = [
  "Fart",
  "Perfume",
  "Pickle",
  "Ferment",
  "Unicornify",
  "Donkeyify",
  "Spaghettify",
  "Reduce",
  "Sneeze",
  "Silence"
];

const difficultyVerbs = [
  "painless",
  "painfull",
  "breeze like",
  "comedic",
  "without a hitch",
  "with many hiccups",
  "smooth",
  "rough",
  "effortless",
  "pulling teeth"
];

const understandAdverbs = [
  "poetic",
  "bend the truth",
  "fuck around and find out",
  "be baffled",
  "figure it out",
  "get lost",
  "comprehend",
  "shit the bed",
  "spell it out",
  "butt scratching"
];

const ages = [
  "six",
  "twelve",
  "nineteen",
  "twenty four",
  "thirty",
  "forty two",
  "sixty nine",
  "eight one",
  "ninety six",
  "one hundred and eleven"
];

Time to try and run this... Need to start some actual files

npx create-react-app fat_lipped_llama
cd fat_lipped_llama
code .