Skip to content

Instantly share code, notes, and snippets.

@keithchambers

keithchambers/jobs.MD

Last active March 22, 2018 03:48
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save keithchambers/4c370c811266720c46f8feb3984e4e0e to your computer and use it in GitHub Desktop.
Save keithchambers/4c370c811266720c46f8feb3984e4e0e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
jobs.MD

package add

Description

Add a package to a repo.

Synopsis

  package add
--package-name <value>
--package-version <value>
--package-repo <value>

Options

--package-name

The name of the package to add to the repo.

--package-version

The version of the package to add to the repo.

--package-repo

The repo to add the package to.

package list

Description

List packages.

Synopsis

  package list
[--package-type <value>]
[--package-repo <value>]

Options

--package-type

Filter by type of package.

Possible values:

  • job
  • service
--package-repo

Filter by package repo.

Output

package-name

The name of the package.

package-version

The version of the package.

package-type

The type of package.

Possible values:

  • jobs
  • services
package-repo

The repo the package is registered with.

package-definition

The URI of the package definition.

Examples

{
  "packages": [
    {
      "package-name": "backup-cassandra",
      "package-version": "3",
      "package-type": "job",
      "package-repo": "repo-1",
      "package-definition": "package://repo-1/backup-cassandra:3"
    }
  ]
}

package describe

Describe a package.

Synopsis

  package describe
--package-name <value>
--package-version <value>
--package-repo <value>

Options

--package-name

The name of the package to describe.

--package-version

The version of the package to describe.

--package-repo

The repo the package is registered with.

Output

(pacakge definition)

package delete

Description

Delete a package from a repo.

Synopsis

  package delete
--package-name <value>
--package-version <value>
--package-repo <value>

Options

--package-name

The name of the package to delete.

--package-version

The version of the package to delete.

--package-repo

The repo to delete the package from.

schedule add

Description

Add a job schedule.

Synopsis

  schedule add
--schedule-name <value>
--schedule-options <value>
--job-name <value>
--job-definition <value>
[--job-options <value>]
--job-queue <value>

Options

--schedule-name

The name of the job schedule.

--schedule-options (json)

The parameters of the job schedule.

--job-name

The name of the job the job schedule submits.

--job-definition

The definition of the job the job schedule submits.

--job-options

The parameters of the job the job schedule submits.

--job-queue

The job queue the job scheduler submits the job to.

schedule list

Description

List job schedules.

Synopsis

  schedule list
--job-queue <value>
[--job-name <value>]

Option

--job-queue

The job queue to query.

--job-name

Filter by the name of the job the schedule submits.

Output

schedule-name

The name of the job schedule.

job-name

The name of the job the job schedule submits.

next-submit-at

The next time the job schedule will submit the job.

last-submit-at

The most recent time the job schedule submitted the job.

last-status

The status of the most recent job submittion.

Possible values:

  • SUBMITTED
  • PENDING
  • RUNNABLE
  • RUNNING
  • SUCCEEDED
  • FAILED
  • CANCELLED
last-succeeded-at

The most recent time the scheduled job returned SUCCEEDED.

last-failed-at

The most recent time the scheduled job returned FAILED.

schedule-enabled

If the job schedule is enabled.

Possible values:

  • TRUE
  • FALSE

Example

{
  "schedules": [
    {
      "schedule-name": "backup-dev-cassandra",
      "job-name": "backup-dev-cassandra",
      "last-status": "SUCCEEDED",
      "last-submit-at": 14804813016253,
      "next-submit-at": 14804919726301,
      "last-succeeded-at": 148048193241,
      "last-failed-at": 148028638541,
      "schedule-enabled": "TRUE"
    }
  ]
}

schedule describe

Description

Describe a job schedule.

Synopsis

  schedule describe
--schedule-name <value>
--job-queue <value>

Options

--schedule-name

The name of the job schedule to describe.

--job-queue

The job queue to query.

Output

schedule-name

The name of the schedule.

schedule-description

A description of the schedule.

schedule-format

The format of the schedule.

Possible values:

  • cron
  • iso8601
schedule

The schedule.

schedule-enabled

If the schedule is enabled.

Possible values:

  • TRUE
  • FALSE
--job-name

The name of the job the job schedule submits.

--job-definition

The definition of the job the job schedule submits.

--job-options

The parameters of the job the job schedule submits.

Example

{
  "schedules": [
    {
      "schedule-name": "nightly-dev-cassandra-backup",
      "schedule-description": "Nightly job to backup of dev cassandra cluster to S3.",
      "schedule-format": "cron",
      "schedule": "0 4 * *  *",
      "schedule-enabled": "TRUE",
      "job-name": "backup-dev-cassandra",
      "job-definition": "package://repo-1/backup-cassandra:3",
      "job-options": {
        "cassandra-nodes": "cassandra-node-1.thisdcos.dcos:9042,cassandra-node-2.thisdcos.dcos:9042",
        "backup-user": "user://eng/dev/cassandra-backup-user",
        "backup-secret": "secret://eng/dev/cassandra-backup-secret",
        "backup-target": "s3://mesosphere.aws.com/backups/dev-cassandra"
      }
    }
  ]
}

schedule delete

Description

Delete a job schedule.

Synopsis

  schedule delete
--schedule-name <value>

Options

--schedule-name

The name of the job schedule to delete.

job submit

Description

Submit a job to a job queue.

Synopsis

  job submit
--job-name <value>
--job-definition <value>
[--job-options <value>]
--job-queue <value>

Options

--job-name

The name for the job.

--job-definition

The definition for the job.

Possible URI schemes:

  • package://[repo]/<package-name>:<package-version>
  • file://[path]/<job-definition>
  • http://<host>[:port][/path]/<job-definition>
  • https://<host>[:port][/path]/<job-definition>
--job-options

Parameters for the job.

--job-queue

The job queue for the job.

Output

job-id

The unique identifier of the job.

job list

Description

List jobs.

Synopsis

  job list
--job-queue <value>
[--job-status <value>]

Options

--job-queue

The job queue.

--job-status

Filter by the status of the job.

Possible values:

  • SUBMITTED
  • PENDING
  • RUNNABLE
  • RUNNING
  • SUCCEEDED
  • FAILED
  • CANCELLED

Output

job-name

The name of the job.

job-id

The unique identifier for the job.

status

The status of the job.

Possible values:

  • SUBMITTED
  • PENDING
  • RUNNABLE
  • RUNNING
  • SUCCEEDED
  • FAILED
  • CANCELLED
service-account

The service account for the job.

submitted-at

The time the job was submitted.

started-at

The time the job started.

ended-at

The time the job ended.

attempts

The number of job attempts.

Example

{
  "jobs": [
    {
      "job-name": "backup-dev-cassandra",
      "job-id": "48172167-3e3f-4e18-bae5-0fb6ee5e7728",
      "status": "SUCCEEDED",
      "service-account": "backup-user",
      "submitted-at": 1480481301253,
      "started-at": 1480481313993,
      "ended-at": 148048193241,
      "attempts": 2
    },
    {
      "job-name": "etl-logs-to-hdfs",
      "job-id": "48172167-3e3f-4e18-bae5-0fb6ee5e7728",
      "status": "FAILED",
      "service-account": "backup-user",
      "submitted-at": 1521556785556,
      "started-at": 1521556785556,
      "ended-at": 152155692321,
      "attempts": 5
    },
    {
      "job-name": "train-credit-risk-model",
      "job-id": "70dfed04-17c3-4dab-99a4-84a99a3bf89a",
      "status": "RUNNING",
      "service-account": "backup-user",
      "submitted-at": 1521556785556,
      "started-at": 1521556882351,
      "attempts": 1
    },
    {
      "job-name": "train-checking-risk-model",
      "job-id": "70dfed04-17c3-4dab-99a4-84a99a3bf89a",
      "status": "RUNNABLE",
      "service-account": "backup-user",
      "submitted-at": 1521556785556
    }
  ]
}

job describe

Description

Description of a job.

Synopsis

  job describe
--job-id <value>
--job-queue <value>

Options

--job-id

The unique identifier of the job to describe.

--job-queue

The job queue of the job to describe.

Output

job-name

The name of the job.

job-id

The unique identifier for the job.

status

The status of the job.

Possible values:

  • SUBMITTED
  • PENDING
  • RUNNABLE
  • RUNNING
  • SUCCEEDED
  • FAILED
  • CANCELLED
service-account

The service account for the job.

submitted-at

The time the job was submitted.

started-at

The time the job started.

ended-at

The time the job ended.

attempts

The number of job attempts.

Example

{
  "jobs": [
    {
      "status": "SUCCEEDED",
      "job-name": "backup-dev-cassandra",
      "job-id": "48172167-3e3f-4e18-bae5-0fb6ee5e7728",
      "job-definition": "package://repo-1/backup-cassandra:3",
      "job-options": {
        "backup-user": "backup-account",
        "backup-secret": "secret://eng/dev/backup-secret",
        "backup-target": "s3://mesosphere.aws.com/backup/dev-cassandra-1"
      },
      "depends-on": [
        {
          "job-name": "send-backup-event-to-pager-duty",
          "job-id": "9595e4aa-3c5a-4ad7-935a-1f983b9ec918"
        }
      ],
      "submitted-by": "schedule://scheduler-1.thisdcos.dcos/eng/dev/nightly-cassandra-backup",
      "submitted-at": 1480481301253,
      "started-at": 1480481313993,
      "ended-at": 148048193241,
      "attempts": [
        {
          "attempt": 1,
          "attempt-id": "96cde48b-fea9-46c0-b3fc-7b4dc9b282e5",
          "status": "FAILED",
          "started-at": 1480481313993,
          "ended-at": 1480481317000
        },
        {
          "attempt": 2,
          "attempt-id": "2de9c6ae-0662-4a67-ae7f-838ec45baadb",
          "status": "SUCCEEDED",
          "started-at": 1480481317001,
          "ended-at": 148048193241
        }
      ]
    }
  ]
}

job cancel

Description

Cancel a job.

Synopsis

  job cancel
--job-id <value>

Options

--job-id

The unique identifier for the job.

job states

  • SUBMITTED The job that has been submitted to the job queue.

  • PENDING The job resides in the job queue and is not yet able to run due to a dependency on another job.

  • RUNNABLE The job has its dependencies defined and waiting to run.

  • RUNNING The job is running.

  • SUCCEEDED The job has finished and exited with code 0.

  • FAILED The job has failed on all attempts and exited with non-zero code.

  • CANCELLED The job was cancelled by a user.

@keithchambers
Copy link
Author

keithchambers commented Mar 22, 2018 •
edited

Flo builds a job-package and submits to Universe (Mesosphere hosted public package-repo)

Dan adds Flo’s job-package to his private package-registry by ...
Step 1: Downloading the job-package from Universe
Step 2: Adding to private package-repo

Dan builds his own job-package and adds to private package-repo.

Boris registers a job-definition from ...

  • job-package in a package-repo (the job-definition is provided with the job-package)
  • local job-definition file (.yaml)

Boris schedules a job from...

  • job-package in a package-repo (the job-definition is provided with the job-package)
  • local job-definition file (.yaml)
  • registered job-definition

Boris runs an ad-hoc job from...

  • job-package in a package-repo (the job-definition is provided with the job-package)
  • local job-definition file (.yaml)
  • registered job-definition
  • job-schedule

Misc:

  • Boris updates a job-schedule
  • Boris deletes a job-schedule
  • Boris disables a job-schedule
  • Boris enables a job-schedule
  • Boris deregisters a job-definition
  • Boris updates a job-definition

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment