Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Docker cloud: PaaS, orchestration, resource-sharing, provisioning

Experimenting with Docker technologies and related supporting infrastructure, as I'm sure many of you are.

Disclaimer: seeking an opinionated architecture, not a wiki (like: https://github.com/veggiemonk/awesome-docker)

In essence, I am building an open-source cloud that:

  • fans-in/out elastically

  • supports multiple public and private cloud providers

  • negotiates resources [no single-use clusters]

  • HA: SQL, NoSQL and NewSQL

    • On that note, "Big Data" support (for at least Hadoop and Spark)

PaaS features, in particular:

  • CI/CD
  • dependent scaling
  • deploy on `git push`, pipelined to tiers—e.g.: dev, UAT, prod—on e.g.: git tags
  • canary releases: test features on small part of cluster—e.g.: to particular customer segment—before deciding to release [multi-]cluster-wide

Planning to write—and have written—a bunch of Python and Go to tie everything together.

Still considering many options, here's what I have so far:

##Provisioning Apache Libcloud

##Service discovery Consul (yes it does more!)

##Load-balancing Vulcand

##Networking Weave

##Resource negotiation Apache Mesos

##Coordination Consul, Zookeeper (for Mesos, will also use Consul), etcd

##Docker orchestration Still deciding. Looking at Marathon, Kubernates and a fair-few others.

##Monitoring Integrate components from many sources (consul health-checks, regular tech [Prometheus, Influx or similar]). Will throw a shiny frontend together with AngularJS.

##Big Data Mesos (with Myriad?)

##HA databases Compose Governor (Postgres)

##PaaS Still deciding. Probably want something Mesos aware like Deis. See also: "Docker orchestration".


I understand that some are working on reference implementations (e.g.: https://github.com/CiscoCloud/microservices-infrastructure). Not planning to be quite so ambitious! - Just a simple one-man endeavour here, and will be open-sourcing + documenting as I go.

Please tear apart my design and proffer suggestions :)

Thanks

@SamuelMarks

This comment has been minimized.

Copy link
Owner Author

commented Apr 28, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.