Ollama and Llama 2 as Drop-in Replacement for OpenAI with Python

ollama-llama2-openai-dropin-replacement.md

Ollama Enables Open Source AI LLM Models

Step 1: Download Ollama

Visit https://ollama.com to download for your system.

Step 2: Download an AI Model like Llama 2

In the ollama library we see all kinds of available models. We'll use Llama 2 and we have a few options:

In the tags we have a few options:

• llama2: Pull: ollama pull llama2, usage: model_to_use="llama2" below.
• llama2:7b: Pull: ollama pull llama2:7b, usage: model_to_use="llama2:7b" below.
• llama2:70b: Pull: ollama pull llama2:70b, usage: model_to_use="llama2:70b" below.

Step 3: Use in with the Python OpenAI Client

pres.py

from openai import OpenAI

client = OpenAI(
    base_url = 'http://localhost:11434/v1',
    api_key='ollama', # required, but unused
)

model_to_use="llama2"
response = client.chat.completions.create(
  model=model_to_use, 
  messages=[
    {
        "role": "system", 
        "content": "You are a professional content creation assistant."},
    {
        "role": "user", 
        "content": "In 1 sentence, what is the best kind of content to create in 2024?"
    },
  ]
)
print(response.choices[0].message.content)

Responds with something like:

In 2024, the most effective and engaging content will be that which leverages cutting-edge technologies like AI, machine learning, and virtual reality to deliver highly personalized, interactive, and immersive experiences for audiences across various platforms.