Here were our stats from the perspective of OpenStack-Infra when we were doing the OSIC Cloud. I used a sample from the month of April 2016 [ http://graphite.openstack.org/dashboard/#OSIC-April ].
The infra quota was capped 1024 VMs and was spread across three different hardware profiles which gave us the ability to dig into any hardware profile and see the infrastructure performance. While OpenStack-Infra was not able to have more than 1024 active VMs at once, their workload was extremely transient; any single VM cloud last as little as 1 second and as long as 1.5 hours.
Quota spread across the hardware profiles:
- s3500 SSD: 256
- s3700 SSD: 256
- 512 15K SAS: 512
There were never times when Infra was not working so it's very hard to say what the build times would have been for 1000 VMs at once however in looking at a 24 Hour sample [ http://graphite.openstack.org/dashboard/OSIC-April-24h ] and focusing on the s3700 SSD profile I can see all build times were <1 minute on average, with no errors. On this day the job rate was roughly 30K VMs per-hour with swings of up to 70K under peek load. If I look at a single hour from the same day [ http://graphite.openstack.org/dashboard/#OSIC-April-1h ] I can see we created "64564" VMs giving us a build rate of roughly "18" per second with every node becoming "ready" to receive work (it could be reached via SSH) within 20 seconds.