Llama3 + Haystack = 💚

llama3-haystack.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "V28",
      "authorship_tag": "ABX9TyOXf+oUI8TvP5ZwFkEZkgYP",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/gist/ZanSara/e2c23aedde5b7dc1bac33e0dc0e26f6d/llama3.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Llama3-8B-Instruct + Haystack = 💚\n",
        "\n",
        "Haystack support the brand new Llama3 family of models right out of the gate.\n",
        "\n",
        "You need to have:\n",
        "- A HuggingFace account and a read token: https://huggingface.co/docs/hub/security-tokens\n",
        "- Access to the model. Request it here: https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct\n",
        "- About 8GB of GPU RAM\n",
        "\n",
        "Once you're ready, download this notebook and have fun!"
      ],
      "metadata": {
        "id": "du1JRmMGP0nz"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "t8zBQcOq85-d"
      },
      "outputs": [],
      "source": [
        "%pip install haystack-ai"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from getpass import getpass\n",
        "\n",
        "os.environ[\"HF_TOKEN\"] = getpass(\"Your HuggingFace Hub token:\")"
      ],
      "metadata": {
        "id": "TgfAgEtH9z-y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from haystack.components.generators import HuggingFaceLocalGenerator\n",
        "\n",
        "llm = HuggingFaceLocalGenerator(model=\"meta-llama/Meta-Llama-3-8B-Instruct\")\n",
        "llm.warm_up()\n",
        "result = llm.run(prompt=\"\"\"\n",
        "  Would you be willing to financially back an inventor who is marketing a device\n",
        "  that she claims has 25 kJ of heat transfer at 600 K, has heat transfer to the\n",
        "  environment at 300 K, and does 12 kJ of work? Explain your answer by examining\n",
        "  the thermodynamic properties of this device, but be concise.\n",
        "\"\"\")\n",
        "\n",
        "# Solution:\n",
        "# The heat transfer to the cold reservoir is Qc=Qh−W=25kJ−12kJ=13kJ, so the\n",
        "# efficiency is Eff=1−QcQh=1−13kJ25kJ=0.48. The Carnot efficiency is\n",
        "# EffC=1−TcTh=1−300K600K=0.50. The actual efficiency is 96% of the Carnot\n",
        "# efficiency, which is much higher than the best-ever achieved of about 70%,\n",
        "# so her scheme is likely to be fraudulent.\n",
        "# From https://phys.libretexts.org/Bookshelves/University_Physics/Exercises_(University_Physics)/Exercises%3A_College_Physics_(OpenStax)/15%3A_Thermodynamics_(Exercises)\n",
        "\n",
        "print(result)"
      ],
      "metadata": {
        "id": "yZy_vL5Q88Gi"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}