Simple way to run llama3 locally

Running_Ollama_locally.md

On Linux

1. Create a Python virtial environment.

    - python3 -m venv ollama
   

2. To install

    - Open Terminal
    - run: curl -fsSL https://ollama.com/install.sh | sh
   

3. Download model of your choice:

     - ollama pull 'MODEL_NAME' #llama3 (Works on 4GB VRAM)
   

4. Create the following script and save test.py

import ollama

# Construct the prompt
full_prompt = f"{context_prompt} What is a blob loss?"

# Generate a response
response = ollama.generate(model='llama3', prompt=full_prompt)

print(response['response'])

5. cd into your terminal and run python test.py