runpod-ollama.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Runpod ollama in 1 minute\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install requests\n",
    "%pip install runpod\n",
    "\n",
    "import requests\n",
    "import runpod"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup the variables\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "runpod.api_key = \"\"\n",
    "\n",
    "GPU_COUNT = 1\n",
    "GPU_NAME = \"NVIDIA GeForce RTX 4090\"\n",
    "CLOUD_TYPE = \"ALL\"  # \"ALL\", \"COMMUNITY\", \"SECURE\"\n",
    "\n",
    "# to print all available GPU names use:\n",
    "# print(runpod.get_gpus())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create the new pod\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pod = runpod.create_pod(\n",
    "    name=\"ollama\",\n",
    "    image_name=\"ollama/ollama:latest\",\n",
    "    start_ssh=True,\n",
    "    support_public_ip=False,\n",
    "    cloud_type=CLOUD_TYPE,\n",
    "    ports=\"11434/http,22/tcp\",\n",
    "    gpu_count=GPU_COUNT,\n",
    "    gpu_type_id=GPU_NAME,\n",
    ")\n",
    "print(pod)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pod_id = pod[\"id\"]\n",
    "ollama_url = f\"https://{pod_id}-{11434}.proxy.runpod.net\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use ollama\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = \"mistral:7b\"\n",
    "response = requests.post(f\"{ollama_url}/api/pull\", json={\"name\": model})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'model': 'mistral:7b', 'created_at': '2024-03-22T22:31:41.382578449Z', 'message': {'role': 'assistant', 'content': \" Learning data science involves gaining a strong foundation in several key areas, including statistics, machine learning, data manipulation and analysis using tools like Excel, Python, or R, and data visualization. Here's a suggested path to help you get started:\\n\\n1. Mathematics and Statistics: Start with the basics of probability theory, linear algebra, calculus, and statistics. These concepts form the foundation for understanding complex algorithms and models used in data science.\\n\\n2. Programming: Choose a programming language like Python or R, which are widely used in data science. Learn the syntax, control structures, functions, and libraries specific to your chosen language.\\n\\n3. Data Manipulation: Familiarize yourself with tools for cleaning, transforming, and aggregating data such as pandas (Python) or dplyr (R). These skills are essential for working with real-world datasets.\\n\\n4. Machine Learning: Learn various machine learning algorithms like linear regression, logistic regression, decision trees, random forests, support vector machines, k-nearest neighbors, and neural networks. Understand the concepts behind each algorithm and how to implement them using libraries such as scikit-learn (Python) or caret (R).\\n\\n5. Data Visualization: Learn techniques for creating effective visualizations of data using tools like Matplotlib (Python), ggplot2 (R), or Tableau. This skill is crucial for communicating insights from your analysis to stakeholders.\\n\\n6. Big Data Technologies: Familiarize yourself with big data technologies such as Hadoop, Spark, and NoSQL databases. These tools are essential for handling large datasets that cannot be processed using traditional methods.\\n\\n7. Projects and Real-World Experience: Apply your knowledge to real-world projects, either through your job or by participating in data science competitions on platforms like Kaggle. This experience will help you build a strong portfolio and improve your skills.\\n\\n8. Continuous Learning: Data science is an ever-evolving field, so it's essential to stay updated with the latest trends, tools, and techniques. Follow industry experts, read blogs, attend conferences, and engage in online communities to expand your knowledge and network.\"}, 'done': True, 'total_duration': 5833936259, 'load_duration': 2288526953, 'prompt_eval_count': 24, 'prompt_eval_duration': 54961000, 'eval_count': 476, 'eval_duration': 3489372000}\n"
     ]
    }
   ],
   "source": [
    "response = requests.post(f\"{ollama_url}/api/chat\",\n",
    "                         json={\n",
    "                             \"model\": \"mistral:7b\",\n",
    "                             \"messages\": [\n",
    "                                 {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
    "                                 {\"role\": \"user\", \"content\": \"What is the best way to learn data science?\"},\n",
    "                             ],\n",
    "                             \"stream\": False,\n",
    "                             \"keep_alive\": \"10m\",\n",
    "                             \"options\": {\n",
    "                                 \"seed\": 0,\n",
    "                                 \"temperature\": 0,\n",
    "                                 \"num_gpu\": 999,  # offload all layers to GPU\n",
    "                             }\n",
    "                         })\n",
    "\n",
    "print(response.json())"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}