Amit-PivotalLabs

Spark on Cloud Foundry

This document describes one means of running a simple Apache Spark cluster on Cloud Foundry. It makes heavy use of Cloud Foundry's container networking features.

You can see an example running at http://spark-ui-proxy.184.73.108.92.xip.io.

Deploy BOSH-Lite on AWS

This cluster was deployed using BOSH-Lite on AWS. Note, this Director cannot be targetted with the new BOSH CLI (see cloudfoundry-attic/bosh-lite#424), but you can use the "old" Ruby CLI just fine. You can use the new CLI for local workflows like manifest interpolation, and then the "old" CLI for remote workflows like deploying and SSH.

Amit-PivotalLabs

BOSH Links: Why and How

Audience: anyone in the BOSH ecosystem, whether you work on something open-source or proprietary

"BOSH Links" is a feature which simplifies how data is shared between BOSH-deployed jobs that need to collaborate with one another (e.g. a web server and its backing database). Like many BOSH things, the whole "BOSH Links" thing can seem counter-intuitive at first, and it may not be clear why things are the way they are. This note hopes to show some of the powerful benefits of BOSH Links, and provide examples and explanations that make things more intuitive.

1. Why?
2. BASIC: A simple problem statement
3. BASIC: Putting the puzzle pieces together
4. ADVANCED: But what about...

Amit-PivotalLabs

package main

import . "time"

type Every Duration

type Do func()

func (schedule Every) Do(schedulable func()) {
	go func() {

Amit-PivotalLabs

• install Homebrew
• brew install things as you need them
• copy the old ~/.gitconfig
• copy the old ~/.bash_profile
• copy the keys from the old ~/.ssh
• copy the old ~/.kube/config
• figure the rest out as you go, it's not worth over-engineering it...

Amit-PivotalLabs

---
director_uuid: DIRECTOR_UUID

compilation:
  workers: 4

meta:
  environment: cf-runtime-bb-1

  stemcell:

Amit-PivotalLabs

Cleanroom

• Do you sometimes want to run performance benchmarks, stress/load tests, or security vulnerability probes against a shared integration environment, or even a production environment?
• Do you worry about polluting these environments, or not leaving any audit trail when things go wrong?

Here are a couple scripts to setup, and later teardown, a cleanroom environment (user, org, space, quota) for doing just these kinds of experiments.

Example

Amit-PivotalLabs

WARNING: The MEGA team has made some changes to the behaviours and requirements of this tool, we should now consider the cf-deployment README to be the official spec.

This tool will generate manifest(s) referencing the appropriate releases and stemcell, and place manifest files in a given directory. It should have informative messaging to STDOUT and STDERR, but it's just for human convenience, it's not as though the output will be redirected to a file to form a deployment manifest, or anything like that.

Interface Specification

Usage and Inputs

./tools/prepare-deployments <aws|bosh-lite|vsphere|openstack> [path_to_config_file]

Amit-PivotalLabs

#!/bin/bash
export PYTHONPATH=/tmp
out=$(python -c "import test; test.update_db($1=$2)")
echo $out