Bosh is a powerful tool to install and manage your Cloud Virtual Machine workloads (VMs), referred to as “deployments”. You can find docs on //bosh.io/docs. Consider using bosh create-env tool to install to a Cloud of your choice a new Bosh Director.
Bosh uses a CPI (Cloud Provider Interface) to talk to the underlying IaaS/Cloud and manage Cloud workloads, just like a computer uses a driver to talk to a printer. CPIs are very similar to the Infrastructure Cloud Providers later implemented in the ClusterAPI technology, but Bosh abstractions apply to any scope of workloads where ClusterAPI is very restrictive, and Bosh concepts map the problem space better than ClusterAPI, reaching better efficiency in the convergence loop towards the expected (healthy) state of workloads.
Director, Stemcell, Release, Deployment, Instance, are words that have a very specific meaning in Bosh context, so we capitalize them here.
Bosh command line interface is implemented as a Golang binary, you'll need a version compiled for your OS and CPU architecture, and just put it in your path.
To start to work with Bosh on your Cloud you’ll need to bootstrap a new Bosh Director VM on your IaaS or Cloud. A Bosh Director is a Bosh server VM with all needed Bosh components to manipulate your Cloud. To bootstrap Bosh on your Cloud you need to use the bosh create-env CLI, you can find details here: //bosh.io/docs/quick-start.
After you have access Bosh installation, you'll need to login to it and find out what clusters does it manage. This paragraph will help you to do it, also it will get you acquainted with basic Bosh terminology. You are highly incouraged to visit page with Bosh docs: //bosh.io/docs
bosh -h - shows you well formated help message on all commands Bosh CLI has. if you need to help on a specific command, run bosh <command> -h.
** bosh envs - lists the known Bosh installations (also referred to as “Bosh environments”) that are known in ~/.bosh/config and have been registered with bosh alias-env previously. A Bosh Environment consists of a Bosh server VM, running a Bosh Director API, with one or more Bosh CPIs, that can run Bosh tasks to manage workload Deployments of Cloud VMs.
** bosh -e <bosh-env> - is a prefix to your Bosh CLI invocation in order to work with a specific Bosh installation. A useful alternative is to set the BOSH_ENVIRONMENT environment variable instead.
** bosh login - login into Bosh director, it will store your target with creds in the ~/.bosh/config YAML file. But this is seldom used. Instead, setting the BOSH_CLIENT, BOSH_CLIENT_SECRET, and BOSH_CA_CERT environment variables is easier and preferred. Be careful not to leave clear-text credentials in your Bash history file when running export BOSH_CLIENT_SECRET=……… interactively. Instead, favor settings these credentials in your ~/.ssh/environment on your client computer, or any other way for SSH to set these environment variable from the client machine, leaving to written trace of any credentials on the Cloud VM that runs the bosh CLI.
** bosh env - shows all actual options of Bosh Director and current connection.
** bosh deployments (or bosh ds --column=name for short) - lists the worload Deployments that are managed by the targeted Bosh Director.
bosh -d <deployment-name> manifest - prints to stdout the current descriptors (often refered to as “manifest”) for the designated workload Deployment. An alternative is to set the BOSH_DEPLOYMENT environment variable, but it's not so useful because you are often working with several different Deployments on a given Bosh Director, so it's more convenient to use -d <deployment-name> on the comand line.
** bosh -d <deployment-name> deploy path/to/manifest.yml - upload the file that contain new description (often refered to as “manifest”) for a workload deployment, and execute the convergence process, to redeploy the workloads towards the new expected state, with as few disruptions as possible.
** bosh vms - shows the VMs for all Deployments that are managed by the Bosh Director. This is an old verb inherited from the Bosh v1 CLI, later replaced by bosh instances, but the cloud-specific VM details here are sometimes necessary for debugging. Scope is often restricted to some Deployment with -d <deployment-name>, as one Bosh task is created for each Deployment to fech its state.
** bosh instances - shows then Bosh Instances (Virtual Machines, mostly) for all Deployments that are managed by the Bosh Director. This is the new Bosh v2 verb, where “Instance” is the new word for designating what used to be called “VMs”, because some CPIs are actually running their “Bosh instances” as Linux Containers or Kubernetes Pods, instead of Virtual Machines. Scope is often restricted to some Deployment with -d <deployment-name>.
bosh stemcells - see the list of all Stemcells that have been uploaded to the Bosh server. Stemcell is an image of OS that Bosh uses to create your VMs. Such images are specific to the CPI, i.e. specific to the underlying IaaS/Cloud. Public Stemcells can be found on (//bosh.io/stemcells)[https://bosh.io/stemcells]).
bosh releases - see the list of all Releases. release is an archive with all sources, binaries, scripts, and config templates to configure and run any service workloads with Bosh.
bosh -d <deployment-name> errands - see the list of errand jobs that are defined (by the underlying Bosh Releases) for the given Deployment. Contrarily to services, that are meant to run forever 24/7, with as few disruptions as possible, errands are one-off jobs that are meant to have a start and an end. Errands are very useful for implementing one-off cluster management automation, or single-use migration tasks.
bosh locks - every time Bosh updates deployments it puts locks on them. If you try to perform modifications on a locked deployment, Bosh will fail with an error. This command allows you to lists these locks, and understand why a given deployment currently is locked.
bosh tasks - see every task/event that occurred in Bosh deployment. By default many “minor” tasks are hidden away from this list. Adding --all allows you to include these in the list, and adding --recent=1000 will be useful to see more thant the default 30 most recent tasks.
bosh cloud-config - see the setup of your Cloud. Indeed, Bosh deployment manifests are agnostic from the underlying Cloud, and thus cloud-config is defining the mapping between Bosh abstractions and their actual Cloud-dependent implementation, as performed by the CPI. Main sections are describing (pre-)existing networks, defining instance (VM) types and variations (referref to as “VM extensions”), disk types, and availability zones. bosh cloud-config is an old Bosh v1 verb though, and the management of Bosh configs has been revisited with Bosh v2 with bosh configs.
bosh configs - is the new command for listing all the configs that Bosh knows, by type and name, along with their versions. There are 3 types: cloud, runtime and cpi configs. Cloud config is explained above. Runtime config are pieces of deployment descriptors (manifests) that are merged on top of the manifests of all Deployments managed by the Bosh server. CPI configs are hodling configuration for CPIs to access the and manage their respective underlying Clouds. This allows one Bosh server to operate deployments spanning over many Clouds.
bosh -d <deployment-name> manifest > path/to/manifest.yml - downloads the description of some Deployment as a file—it's useful.
Bosh is a great tool to run your clustered workloads… This paragraph shows how to run your deployments with Bosh.
** bosh update-cloud-config path/to/cloud-cfg.yml - legacy Bosh v1 verb to update a single cloud config named default. With Bosh v1 this was relevant, but with Bosh v2 you should check for any other piece of cloud config that may also apply.
bosh update-config --type=cloud --name=default path/to/cloud-cfg.yml - equivalent to the above, but using the new Bosh v2 multi-configs semantics. This redefines a set of mapping of Bosh concepts to actual Cloud implementations, named default.
bosh upload-stemcell <stemcell-path-or-url> - upload a Stemcell to the Bosh Director; you can pass this command a path to a Stemcell tarball that will be uploaded to the Director, or a URL to a Stemcell tarball that the Director will download is a dedicated task. Stemcells are large files and the Director often has higher bandwidth than your client computer, so it's often efficient to let the Director download its Stemcells.
bosh upload-release <release-path-or-url> - upload a Bosh Release to Bosh Director; you can pass this command a path to a file with release or URL, where the Bosh Release is to be fetched by the Director. Rememeber that from the Cloud, the Director often has much higher bandwidth than your client machine.
** bosh -d <depl-name> deploy -n path/to/manifest.yml - deploy changes to the current cluster; where -n means non-interactive mode. The CLI will display a diff of the expected final state (healthy, hopefully), then immediately execute the convergence process without asking for any confirmation.
** bosh ssh <instance-name>/<instance-index> - ssh to a Bosh Instance.
bosh logs <instance-name>/<instance-index> - fetch tarball with logs for specific Bosh Instance.
bosh -d <depl-name> run-errand <errand-name> - run the specified errand for a given Deployment.
bosh task <task-number> - display task status and start tracking its output. Ctrl-C to detatch from the output.
bosh cancel-task <task-number> - stop task once it reaches the next checkpoint.
Bosh not only deploys your clustered workloads on the Cloud, but also provides support for managing their lifecycle. This paragraph shows main commands that help you to maintain and support your cluster.
bosh -d <depl-name> cloud-check - very useful command if something went wrong, forces Bosh to compare desired state of your infrastructure and the one you actually have. After Bosh finds any mismatch, it will try to recover this error providing you with possibility to choose action to recover. When you know in advance which type of remediation you need, there are options to this command to instruct Bosh to apply these without asking.
bosh -d <depl-name> instances --ps - when bosh cloud-check is of no help, this gives a detailed status of all Instances in the given Deployment. A very important hint!
bosh -d <depl-name> stop -n --hard <instance-name>/<instance-index> - poweroff a given Instance so that it doesn't get in the way of recreating another one in the same cluster.
bosh -d <depl-name> restart <instance-name>/<instance-index> - restarts a Bosh Instance.
bosh -d <depl-name> recreate <instance-name>/<instance-index> - destroys and recreates a Bosh Instance. This is usefull to eliminate any human-introduced mis-configuration.
bosh cleanup - removes all unused Releases and Stemcells.
bosh ssh <job-name>/<job-index> - Connects to a Bosh Instance.
sudo -i - recommended when connecting to a Bosh Instance, so that you get monit in your $PATH!
watch "monit summary && echo && tail /var/vcap/monit/monit.log && echo && find /var/vcap/sys/log -type f -ls" - Watch for monit status, latest monit activity, and log files activity. Useful when you try to spot an issue without knowing exactly which one in advance.
[ -z "$(command -v jq)" ] && apt-get -y install jq; jq -r '"\(.job.name)/\(.index)"' /var/vcap/bosh/spec.json - To know exactly on which instance you are connected to. Very useful when you have dozens of SSH connections open!
find /var/vcap/sys/log -type f -ls - Lists all log files for all jobs, so that you can select one to look at.
tail -F $(find /var/vcap/monit/monit.log /var/vcap/sys/log -type f) - Follow all log files on the instance. Also know as the “mega tail”.
To use Bosh you'll need to create Releases, Bosh community has plenty of ready-to-use Releases, referenced in here //bosh.io/releases.
If consider to create your own Bosh Release, we advise you to follow the recommendations and examples from the Exemplar Release (and don't use the bosh-gen Gem that is unfortunately generating way too much unnecessary boilerplace code)
** bosh init-release --git - to initialize the a repository for a new Bosh Release. The name used will be the directory name (final part of pwd), to be adjusted in config/final.yml soon, as it will be very hard to change after you have cut your first final release.
** bosh create-release --with-tarball - run this command in the folder of your Bosh Release to generate a Bosh Release tarball. This will create “dev release” for testing, and not a final one for general availability.
bosh add-blob <local_path> [<blob-dir>] - add a blob (i.e. Binary Large OBject, often an archive containing binaries or source code) to your Release project.
bosh blobs - list of blobs that are already added to the Release (i.e. listed in config/blobs.yml).
Disks are vital in Bosh Deployments to store and share state between jobs.
bosh disks --orphaned - see all disks that are not attached to any VMs. These are kept around for 5 days in case an accidental “detach” operation would need to mount then to a new VM and get their data back.
bosh attach-disk <instance-name>/<instance-index> <disk-cloud-id> - attach a disk to an Instance. (Attached disks are mounted to /var/vcap/store and when connected to an instance, they are visible with mount.)