Experimenting with Docker technologies and related supporting infrastructure, as I'm sure many of you are.
Disclaimer: seeking an opinionated architecture, not a wiki (like: https://github.com/veggiemonk/awesome-docker)
In essence, I am building an open-source cloud that:
-
fans-in/out elastically
-
supports multiple public and private cloud providers
-
negotiates resources [no single-use clusters]
-
HA: SQL, NoSQL and NewSQL
- On that note, "Big Data" support (for at least Hadoop and Spark)
PaaS features, in particular:
- CI/CD
- dependent scaling
- deploy on `git push`, pipelined to tiers—e.g.: dev, UAT, prod—on e.g.: git tags
- canary releases: test features on small part of cluster—e.g.: to particular customer segment—before deciding to release [multi-]cluster-wide
Planning to write—and have written—a bunch of Python and Go to tie everything together.
Still considering many options, here's what I have so far:
##Provisioning Apache Libcloud
##Service discovery Consul (yes it does more!)
##Load-balancing Vulcand
##Networking Weave
##Resource negotiation Apache Mesos
##Coordination Consul, Zookeeper (for Mesos, will also use Consul), etcd
##Docker orchestration Still deciding. Looking at Marathon, Kubernates and a fair-few others.
##Monitoring Integrate components from many sources (consul health-checks, regular tech [Prometheus, Influx or similar]). Will throw a shiny frontend together with AngularJS.
##Big Data Mesos (with Myriad?)
##HA databases Compose Governor (Postgres)
##PaaS Still deciding. Probably want something Mesos aware like Deis. See also: "Docker orchestration".
I understand that some are working on reference implementations (e.g.: https://github.com/CiscoCloud/microservices-infrastructure). Not planning to be quite so ambitious! - Just a simple one-man endeavour here, and will be open-sourcing + documenting as I go.
Please tear apart my design and proffer suggestions :)
Thanks
Originally posted: https://groups.google.com/d/topic/docker-user/1nZVAF2OiwA