ljfranklin

What is this?

Some ad-hoc performance testing on cf push. Variables are number of concurrent pushes, number of diego cells, CPU count of API VMs, while the number of API VMs is fixed at one. Disabled resource matching as the test script pushes the same app each time.

Main Findings:

• A single cf push takes ~30-32 seconds regardless of number of cores, cells, or workers
• The number of diego cells is the main limiter as concurrency increases
  • 32 concurrent pushes took ~59 seconds on average with 1 CC, 4 cores on API, 2 local workers, 4 diego cells
• Dropped to ~42 seconds on average by scaling cells from 4 to 8 VMs

ljfranklin

Chapter 10 - 15 of Site Reliability Engineering

Possible discussion topics:

• Thoughts on this recommendation: Being on-call should strike a balance between quantity (percent of time spent doing on-call activities) and the quality (number of incidents that occurred while on-call).

  • Quantity: Spend at least 50% of time doing engineering, no more that 25% of remainder should be on-call
  • Quality: If too many incidents occur on a given on-call shift, the SRE will not have time to properly perform the incident response responsibilities such as: root-cause analysis, remediation, and follow-up activities like writing a postmortem and fixing bugs. Google found these activities take ~6 hours on average, so there is a max of 2 incidents per 12 hour shift of on-call.

• One Emergency Response recommendation was to intentionally break your systems to see if they fail in the way you expect.

• Anyone want to share experiences of doing this at Pivotal?

ljfranklin

<%
# current approach
cc_uploader_url = nil
if_link("cc_uploader") do |cc_uploader|
  cc_uploader_url = "http://#{cc_uploader.p("internal_hostname")}:#{cc_uploader.p("http_port")}"
end

if cc_uploader_url.nil?
  cc_uploader_url = p("cc.diego.cc_uploader_url")
end

ljfranklin

Keybase proof

I hereby claim:

• I am ljfranklin on github.
• I am ljfranklin (https://keybase.io/ljfranklin) on keybase.
• I have a public key whose fingerprint is C15F 116B DAA9 AC25 0E20 2752 9486 66B3 EEC6 A0F0

To claim this, I am signing this object:

ljfranklin

2016/05/19 05:18:37 [INFO] Terraform version: 0.6.16  6e586c89394f5474b6c5f3217a4a75ce80c3fd6c
2016/05/19 05:18:37 [DEBUG] Detected home directory from env var: /root
2016/05/19 05:18:37 [DEBUG] Discovered plugin: atlas = /usr/local/bin/terraform-provider-atlas
2016/05/19 05:18:37 [DEBUG] Discovered plugin: aws = /usr/local/bin/terraform-provider-aws
2016/05/19 05:18:37 [DEBUG] Discovered plugin: azure = /usr/local/bin/terraform-provider-azure
2016/05/19 05:18:37 [DEBUG] Discovered plugin: azurerm = /usr/local/bin/terraform-provider-azurerm
2016/05/19 05:18:37 [DEBUG] Discovered plugin: chef = /usr/local/bin/terraform-provider-chef
2016/05/19 05:18:37 [DEBUG] Discovered plugin: clc = /usr/local/bin/terraform-provider-clc
2016/05/19 05:18:37 [DEBUG] Discovered plugin: cloudflare = /usr/local/bin/terraform-provider-cloudflare
2016/05/19 05:18:37 [DEBUG] Discovered plugin: cloudstack = /usr/local/bin/terraform-provider-cloudstack

ljfranklin

2016/05/19 04:48:48 [INFO] Terraform version: 0.6.16  6e586c89394f5474b6c5f3217a4a75ce80c3fd6c
2016/05/19 04:48:48 [DEBUG] Detected home directory from env var: /root
2016/05/19 04:48:48 [DEBUG] Discovered plugin: atlas = /usr/local/bin/terraform-provider-atlas
2016/05/19 04:48:48 [DEBUG] Discovered plugin: aws = /usr/local/bin/terraform-provider-aws
2016/05/19 04:48:48 [DEBUG] Discovered plugin: azure = /usr/local/bin/terraform-provider-azure
2016/05/19 04:48:48 [DEBUG] Discovered plugin: azurerm = /usr/local/bin/terraform-provider-azurerm
2016/05/19 04:48:48 [DEBUG] Discovered plugin: chef = /usr/local/bin/terraform-provider-chef
2016/05/19 04:48:48 [DEBUG] Discovered plugin: clc = /usr/local/bin/terraform-provider-clc
2016/05/19 04:48:48 [DEBUG] Discovered plugin: cloudflare = /usr/local/bin/terraform-provider-cloudflare
2016/05/19 04:48:48 [DEBUG] Discovered plugin: cloudstack = /usr/local/bin/terraform-provider-cloudstack

ljfranklin

## Bash Prog Intro Notes

http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
http://tldp.org/LDP/abs/html/
http://mywiki.wooledge.org/BashPitfalls

## Intro:
I feel like writing bash is like building something with hot glue.
When try it for the first time, you'll probably make a big mess and burn yourself.
As you get a little more experienced, you can slap things together really quickly but the end result won't be very sturdy or pretty.

ljfranklin

Links:

• Bosh-lite on AWS
• Deploy CF on bosh-lite
• AWS CLI

Setup AWS Infrastructure:

1. Create VPC
2. Create subnet
3. Create IGW and bind to subnet
4. Create bosh security group (see bosh aws link for ports)