Submit a job to Paperspace Gradient using the Python API

PaperspaceSubmitJob.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import paperspace.jobs\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us first define the parameters that we need. The *workspace* is a directory which will be zipped and transfered to the server, the unzipped directory will then be the working directory for the job. The *machineType* determines the machine type on which the job will be place, C2 is a CPU only machine, K80, P100 etc. are some GPU enabled machines. The *command* is the command that will be executed, i.e. the actual job. The *container* specifies the container image that will be used to assemble the job. Finally, you have to declare a *project name*, which is later used when, for instance, you want to list the running jobs using the CLI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'command': 'pwd ; ls -l; cat amIThere.txt',\n",
       " 'container': 'paperspace/tensorflow-python',\n",
       " 'machineType': 'C2',\n",
       " 'project': 'MachineLearning',\n",
       " 'workspace': '/tmp/pstest'}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#\n",
    "# Define parameters\n",
    "#\n",
    "params = {}\n",
    "params['workspace'] = \"/tmp/pstest\"\n",
    "params['machineType'] = \"C2\"\n",
    "params['command'] = \"pwd ; ls -l; cat amIThere.txt\"\n",
    "params['container'] = 'paperspace/tensorflow-python'\n",
    "params['project'] = \"MachineLearning\"\n",
    "params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our simple job will just list the content of the current directory and print out the content of a file amIThere.txt to demonstrate that this file is submitted along with the job. Hence we first create that file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "if not os.path.exists(\"/tmp/pstest\"):\n",
    "    os.mkdir(\"/tmp/pstest\")\n",
    "with open(\"/tmp/pstest/amIThere.txt\", \"w\") as myFile:\n",
    "    print(\"Here I am!\", file=myFile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we invoke the paperspace package and use the module jobs to create a job. The parameter *no_logging* ensures that the call returns immediately after job creation and will not wait for the streamed logs to arrive. Note that this assumes that you have logged in first using the CLI, this will store the API key that we use in a JSON file in ~/.paperspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "job = paperspace.jobs.create(params, no_logging=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally let us see what we got back"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Started job with jobId  jsudf23wn0scej\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'artifactsDirectory': '/artifacts',\n",
       " 'cluster': 'PS Jobs',\n",
       " 'container': 'paperspace/tensorflow-python',\n",
       " 'dtCreated': '2018-04-12T07:09:56.032Z',\n",
       " 'dtDeleted': None,\n",
       " 'dtFinished': None,\n",
       " 'dtModified': '2018-04-12T07:09:56.032Z',\n",
       " 'dtProvisioningFinished': None,\n",
       " 'dtProvisioningStarted': None,\n",
       " 'dtStarted': None,\n",
       " 'dtTeardownFinished': None,\n",
       " 'dtTeardownStarted': None,\n",
       " 'entrypoint': 'pwd ; ls -l; cat amIThere.txt',\n",
       " 'exitCode': None,\n",
       " 'id': 'jsudf23wn0scej',\n",
       " 'jobError': None,\n",
       " 'machineType': 'C2',\n",
       " 'name': 'job 64',\n",
       " 'parentJobId': None,\n",
       " 'project': 'MachineLearning',\n",
       " 'projectId': 'pre076cyh',\n",
       " 'queuePosition': '1',\n",
       " 'seqNum': 64,\n",
       " 'startedByUserId': 'u0q3drsn',\n",
       " 'state': 'Pending',\n",
       " 'usageRate': 'C2 Hourly',\n",
       " 'workingDirectory': '/paperspace',\n",
       " 'workspaceUrl': 's3://ps-projects/pre076cyh/jsudf23wn0scej/pstest.zip'}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "jobId = job['id']\n",
    "print(\"Started job with jobId \", jobId)\n",
    "job"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}