README.md

autograde

A online grading microservice

What is grading?

Grading in this case is:

1. Take a given payload (notebooks, code files, url with zip files, etc)
2. Run it in a given environment (docker image), with possibly some ancillary files
3. Record and 'return' the STDOUT / STDERR to caller

This should be done in a scaleable and secure fashion.

How is the payload given to the service?

It can support both files passed in as HTTP mutli-part uploads, or a url that provides the payload.

How is the result communicated back to the caller?

Options would be:

1. WebHook notification
2. Polling on the caller's side

What are 'ancillary files'?

These are things that are copied into the environment before being executed. These should provide the grading scripts.

Currently these can be pulled in from a git repository.

What is all the state that needs to be stored?

1. Payload content (minio / GCS)
2. Grading information (Bunch of files!)
3. Webhook metadata (where to send results to + additional metadata)
4. In-flight requests (redis?)

Architecture

1. Flask frontend, accepts POSTs: a. URL to get payload from b. environment to run tests in (docker image spec) c. ancillary files to pull in (URL?) d. arbitrary metadata to be associated with this request e. response webhook URL
2. Puts in queue
3. Worker picks it up, fetches payload + ancillary files if necessary.
4. Runs a docker container of environment, mounting ancillary files + payload in it appropriately. Captures stdout / stderr
5. Worker fires off webhook to target

Other requirements

1. Should be able to completely run locally, with just docker daemon (and possibly redis) as dependency