example notebook for deploying GPT-Neo in AWS Sagemaker

deploy_gpt_neo_on_sagemaker.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "80784592",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sagemaker>=2.48.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (2.59.1)\n",
      "Requirement already satisfied: google-pasta in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (0.2.0)\n",
      "Requirement already satisfied: protobuf3-to-dict>=0.1.5 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (0.1.5)\n",
      "Requirement already satisfied: importlib-metadata>=1.4.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (3.7.0)\n",
      "Requirement already satisfied: smdebug-rulesconfig==1.0.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (1.0.1)\n",
      "Requirement already satisfied: pathos in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (0.2.8)\n",
      "Requirement already satisfied: packaging>=20.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (20.9)\n",
      "Requirement already satisfied: boto3>=1.16.32 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (1.18.28)\n",
      "Requirement already satisfied: attrs in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (20.3.0)\n",
      "Requirement already satisfied: pandas in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (1.1.5)\n",
      "Requirement already satisfied: protobuf>=3.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (3.17.3)\n",
      "Requirement already satisfied: numpy>=1.9.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from sagemaker>=2.48.0) (1.19.2)\n",
      "Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from boto3>=1.16.32->sagemaker>=2.48.0) (0.10.0)\n",
      "Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from boto3>=1.16.32->sagemaker>=2.48.0) (0.5.0)\n",
      "Requirement already satisfied: botocore<1.22.0,>=1.21.28 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from boto3>=1.16.32->sagemaker>=2.48.0) (1.21.28)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.25.4 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from botocore<1.22.0,>=1.21.28->boto3>=1.16.32->sagemaker>=2.48.0) (1.26.6)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from botocore<1.22.0,>=1.21.28->boto3>=1.16.32->sagemaker>=2.48.0) (2.8.1)\n",
      "Requirement already satisfied: typing-extensions>=3.6.4 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from importlib-metadata>=1.4.0->sagemaker>=2.48.0) (3.7.4.3)\n",
      "Requirement already satisfied: zipp>=0.5 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from importlib-metadata>=1.4.0->sagemaker>=2.48.0) (3.4.0)\n",
      "Requirement already satisfied: pyparsing>=2.0.2 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from packaging>=20.0->sagemaker>=2.48.0) (2.4.7)\n",
      "Requirement already satisfied: six>=1.9 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from protobuf>=3.1->sagemaker>=2.48.0) (1.15.0)\n",
      "Requirement already satisfied: pytz>=2017.2 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pandas->sagemaker>=2.48.0) (2021.1)\n",
      "Requirement already satisfied: pox>=0.3.0 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pathos->sagemaker>=2.48.0) (0.3.0)\n",
      "Requirement already satisfied: dill>=0.3.4 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pathos->sagemaker>=2.48.0) (0.3.4)\n",
      "Requirement already satisfied: ppft>=1.6.6.4 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pathos->sagemaker>=2.48.0) (1.6.6.4)\n",
      "Requirement already satisfied: multiprocess>=0.70.12 in /home/ec2-user/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages (from pathos->sagemaker>=2.48.0) (0.70.12.2)\n",
      "\u001b[33mWARNING: You are using pip version 21.1.3; however, version 21.2.4 is available.\n",
      "You should consider upgrading via the '/home/ec2-user/anaconda3/envs/pytorch_latest_p36/bin/python -m pip install --upgrade pip' command.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install \"sagemaker>=2.48.0\" --upgrade"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f7afcce",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.huggingface import HuggingFaceModel\n",
    "import sagemaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "bbf50838",
   "metadata": {},
   "outputs": [],
   "source": [
    "role = sagemaker.get_execution_role()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ca28f8ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Hub Model configuration. https://huggingface.co/models\n",
    "hub = {\n",
    "  'HF_MODEL_ID':'EleutherAI/gpt-neo-1.3B', # model_id from hf.co/models\n",
    "  'HF_TASK':'text-generation' # NLP task you want to use for predictions\n",
    "}\n",
    "\n",
    "# create Hugging Face Model Class\n",
    "huggingface_model = HuggingFaceModel(\n",
    "   env=hub,\n",
    "   role=role, # iam role with permissions to create an Endpoint\n",
    "   transformers_version=\"4.6\", # transformers version used\n",
    "   pytorch_version=\"1.7\", # pytorch version used\n",
    "   py_version=\"py36\", # python version of the DLC\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "fa329aae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------!"
     ]
    }
   ],
   "source": [
    "# deploy model to SageMaker Inference\n",
    "predictor = huggingface_model.deploy(\n",
    "   initial_instance_count=1,\n",
    "   instance_type=\"ml.p3.2xlarge\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5436cb6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'generated_text': ' A method to record your meetings and keep them organized \\n Post'}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictor.predict({\n",
    "\t'inputs': \"Write a short, thought-provoking, and eye-catching post title for Hacker News (https://news.ycombinator.com/) submission based on a blog post’s provided description. \\n Post description: The post describes how to deliver constructive feedback in difficult situations. The author claims that thoughtful, empathetic language can make or break business relationships and suggests that excellent communication is not just about what you say; it is about what other people hear. \\n Post title: A guide to difficult conversations \\n Post description: The post describes applying the behavior model invented by Stanford professor BJ Fogg to battle procrastination. The author suggests the three things we need to change the behavior and explains why we procrastinate. \\n Post title: How to stop procrastinating by using the Fogg Behavior Model \\n Post description:  The post explains that conventional education does not teach students how to use tools like the command line, text editors, and control systems. The idea is that programmers spend hundreds of hours using these tools and therefore need to master them to work more efficiently. \\n Post title: The missing semester of CS education \\n Post description: The post describes the process for taking thoughtful meeting notes. The author covers how he trained himself to remember what is happening during long meetings and suggests actionable steps to integrate the note-taking habit in your life. \\n Post title:\",\n",
    "    'parameters': {\n",
    "        'max_length': 300,\n",
    "        'temperature':0.8,\n",
    "        \"return_full_text\": False,\n",
    "    }\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "d45f6198",
   "metadata": {},
   "outputs": [],
   "source": [
    "# predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "conda_pytorch_latest_p36",
   "language": "python",
   "name": "conda_pytorch_latest_p36"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}