events thoughts

gistfile1.md

Data that you get with "start job"

Current (from strider/lib/jobs.js)

- user_id
- github_apikey
- job_id
- repo_config (unsanitized)
- deploy_config
- deploy_target
- repo_ssh_url
- job_type
- github_commit_info

What I'd like to see

- user_id
- repo: {
    url: github.com/jaredly/jared.git
    auth: {
      type: https
      username:
      password:
    } || {
      type: ssh
      key:
    }
    provider: github || bitbucket || gitlab
    vcs: git || hg
  }
- job_id
- ref: {
    branch:
    id:
  } || {
    fetch: "+refs/pull/141/merge"
  }
- job_type (TEST, DEPLOY, TEST&DEPLOY)
- trigger [see trigger spec below]

// stuff gotten from the DB & config file. Any branch-specific config
// is already factored in.
- plugins: [names, in, order]
- config {
    .. defined by the plugins ..
  }

Trigger spec

{
  type:
  author: {
    id: <strider uid>
    url: <ex: github.com/username>
    name: "Jared Forsyth"
    email: [optional]
    gravatar: [suggested]
    username: [only applicable for github, etc.]
  }
  message: [displayed in the ui]
  timestamp: [required]
  url: [message links here]
  source: {
    type: UI
    page: dashboard
  } || {
    type: API
    app: [app id? app name?]
  } || {
    type: plugin
    plugin: name
  }
}

The only trigger provided by default is "manual".

Ideas for other triggers:

• commit [by the github plugin]
• pull-request [by the github plugin]
• dependency [dependency-checker]

## Events

### Events the queen listens for

• queue:new {job data}

The queen then decides which drone should handle the request. This is
the one that's fastest and has open capacity. If all are full, then
just the one with the shortest queue (relative to its capacity).

### Events the drone listens for

• job:query-info jobid

This will generally get fired by the `api/jobs` endpoint if there are
jobs in progress. That way the user will be able to see the full
output of a running job when they get to the page, not just the output
since they showed up.

### Events the drone will fire

• browser eventtype, [args] // proxying job:* events from the remote
• job:info jobid, job-data // in response to job:query-id

### Events the drone will fire to the remote

• drone:query-info
• queue:new {job data}

  
### Events the remote fires to the drone

• drone:info {speed: int, capacity: int} // maybe report plugins as well? See thoughts @ bottom

#### command specific

• job:cmd:start id, num, command, screencmd [sanitized version of command]
• job:cmd:done id, num, exitCode
• job:cmd:stdout id, num, text
• job:cmd:stderr id, num, text

#### plugin specific

Currently these are only sent up to the browser. They aren't
propagated by the drone on the main strider server. Do we want this?

• job:plugin id, plugin, [whatever the plugin passes in]

#### general

They would be output as part of whatever command is currently being
run.

• job:stdout id, text
• job:stderr id, text

#### status

• job:queued id, timestamp
• job:started id, timestamp
• job:tested id, code, timestamp
• job:deployed id, code, timestamp
• job:done id, timestamp

# Thoughts

Currently all drones are expected to have all plugins. Is that an
issue? Drones could of course send in `drone:info` which plugins are
installed, and the queen would then only send it jobs it could
handle. But is there a real use case for that?

Gonna comment as I think.

The fact we're talking about Queens and Drone's makes me think this should be 2 modules - one to manage job creation (the queen) and one to actually setup the environment and run tasks (drone)

So rather than a list of plugins and config, I think it makes sense to give the full plugin/pluginconfig array from the repo out - this has enabled/disabled state and is isomorphic to what will be in the flat file.

so how about 2 modules: (I'm renaming a bunch of stuff on the fly here)

strider-job-queue (Queen) ( perhaps strider-zeromq-job-queue ?)

• I think there is value in a separate job queue module. Possibly the ability for different job queues - we may want a job queue that can handle network jobs, or local ones, or ones in a docker VM.

Questions:

• How tightly is this coupled to the type of strider-job?
• Can the strider-job just import this maybe?
• or does strider somehow specify which type of strider-job we're using?

strider-job - a family of modules that do work on an environment. (Drone)

ie. strider-simple-job, strider-docker-job etc.

• We should specify a concrete interface for these, so they can easily be developed against that interface. I don't think the flexibility argument flies here - to be able to develop against an interface, a standard set of methods must be known. Because of this, I don't like the event emitter pattern, I feel as though strider-jobs should just have a set of known methods available on module.exports, so you can require any strider-job and run eg. .start(repo, command, data, pty, cb)etc.

(Perhaps there is an event emitter specified as part of this for communication back etc, but I think we really want to define this pretty strictly)

Because all of the repo's plugins need to be installed in whatever the environment we run the strider-job in, we need a way to pass plugin installation through to the environment setup. For npm modules this is easy - we can just npm install them. But what about proprietary plugins that aren't in the npm registry.

@peterbraden take a look at strider-worker-core. I think it's exactly along the lines you were thinking of with strider-job.

re: proprietary plugins ... we can still just use npm install. For private git repos:

 npm install git+https://oauthtoken@github.com/repo/name.git

Or anything else

 npm install https://username:password@example.com/node_module.tgz