Skip to content

Instantly share code, notes, and snippets.

@ollyg
Last active November 21, 2019 18:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ollyg/248289ad8c1f41aefde22e9663f6c18d to your computer and use it in GitHub Desktop.
Save ollyg/248289ad8c1f41aefde22e9663f6c18d to your computer and use it in GitHub Desktop.
netdisco3 ideas

Selected Technologies

  • FastAPI
  • TaskTiger
  • Redis
  • PostgreSQL
  • peewee
  • structlog

Interesting Software

  • React Bootstrap
  • Prometheus
  • Grafana
  • etcd
  • YAML
  • Napalm ??
  • React
  • PostgreSQL
  • structlog
  • APScheduler
  • peewee
  • supervisor superlance

TODO

A scheduler with a good dashboard, which isn't dependant on a framework we're not using (Django, etc).

List of features to have or implement:

  • tasks run forked
  • single instance of task in queue
  • lock similar tasks
  • retry exceptions all or select, with timers
  • random selection from queues
  • periodic tasks
  • structured logging

Possibilities:

  • tasktiger has all the features we need
    • Redis only
    • dashboard uses flask-admin
    • full featured
    • code seems a little... undeveloped?
  • dramatiq
    • RabbitMQ or Redis
    • can pipeline actors easily
    • bit confusing terminology
    • can age and time out tasks
  • Mr. Q has a nice dashboard
    • Needs Redis and MongoDB
    • has class rather than decorator approach?
    • interesting mapping of cli params to args
    • only greenlets
    • docs seem thin
  • huey2 has no dashboard
    • Redis or sqlite (and possibly postgresql)
    • cannot age or time out tasks
    • seems to have no queue introspection

Related interesting projects:

  • APScheduler

Data Model

Data is stored in the database according to the model (YANG).

graph LR
A(discovery) --> B(interface)
B -- which emits --> C(YANG)
C --> D(database)

Use of database views to create the objects required for the API.

graph LR
A(database) --> B(view)
B --> C(API)
C --> D(frontend)

Written with StackEdit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment