Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
BOSH Links: Why and How
@johannaratliff

This comment has been minimized.

Copy link

commented Mar 7, 2017

Hi Amit,

We were noticing that the example files currently contain an empty relevant_later link. Later, the username, password, and port properties are referenced while mentioning that the max_connections property is not exposed. It seems that currently none of the properties are exposed and require

provides:
  - name: relevant_later
    type: relevant_later
    properties:
    - username
    - password
    - port

to be available.

@robdimsdale

This comment has been minimized.

Copy link

commented Mar 7, 2017

Good use of gifs. I like the idea of break time if this is being structured as a course. As a (static) docs page it might make more sense to divide it up into "tutorial: getting started" and "tutorial: advanced" and let the reader decide how to manage their time.

I don't think I saw anything incorrect, but it was a little hard to parse. There's a large amount of information density, so perhaps consider breaking up the multi-sentence paragraphs into smaller ones? I know it's subjective, but it would probably improve my experience.

Some minor feedback below:

Remove the main function from the golang analogy for importing - the import () section is sufficient to get the point across.

Minor copy changes e.g.:

Removing words which might offend the user like "obviously"

introducing some problems with manifests that obviously need solving

might better read:

introducing some problems with manifests that need solving

Redundant usage of words like "basic":

The basic problems raised in the "BASIC" sections could be solved without Links

might be better as:

The problems raised in the "BASIC" sections could be solved without Links

Spurious use of first-person tense:

but let me bring up a few questions which we'll address after

could be anything else that doesn't insert an individual into the narrative.

@Amit-PivotalLabs

This comment has been minimized.

Copy link
Owner Author

commented Mar 8, 2017

Thanks @JohannaSmith, I fixed that omission.

As far as readability, and whether it helps improve your understanding of links, do you have any feedback? If you were already familiar with links, do you have a colleague who is less familiar, or could you imagine yourself in those shoes?

Thanks @robdimsdale, will incorporate that feedback.

@dsabeti

This comment has been minimized.

Copy link

commented Mar 16, 2017

This doc is fantastic. I thought the following were really helpful:

  • Explaining the difference between the name and type of a link. I've seen a decent amount of confusion about this when release teams implement links.
  • Explaining some of the current shortcomings of links -- for example, the way that property names still need to be synchronized in different releases.

Thank you webmaster! But our database_process is in another deployment!

Brilliant.

@drnic

This comment has been minimized.

Copy link

commented Mar 19, 2017

I'm trying to convert the etcd-release and its manifests to links

When I deploy I get a "Multiple instance groups provide links" error:

  - Multiple instance groups provide links of type 'consul'. Cannot decide which one to use for instance group 'etcd'.
     etcd.consul.consul_agent.consul
     etcd.etcd.consul_agent.consul

The two instance groups include the consul_agent job:

instance_groups:
- name: consul
  jobs:
  - name: consul_agent
    release: consul
- name: etcd
  jobs:
  - name: consul_agent
    release: consul
  - name: etcd
    release: etcd

This topic is solved in https://gist.github.com/Amit-PivotalLabs/c39528248b8cdc4ba8e347f8aa68abb6#what-happens-if-multiple-things-provide-the-same-type-of-link I think; but perhaps mention the above error message in the text to help googling find the solution.

For the example above, I changed it to:

instance_groups:
- name: consul
  jobs:
  - name: consul_agent
    release: consul
    provides:
      consul: {as: consul_server}
    consumes:
      consul: {from: consul_server}
- name: etcd
  jobs:
  - name: consul_agent
    release: consul
    consumes:
      consul: {from: consul_server}
  - name: etcd
    release: etcd
@drnic

This comment has been minimized.

Copy link

commented Mar 20, 2017

I tried to consume a link from another deployment but the error message suggests I'm getting it wrong.

I am deploying eats (acceptance tests for etcd), where I try to reference the aliased consul_server link in the message above:

instance_groups:
- name: acceptance-tests
  jobs:
  - name: acceptance-tests
    release: etcd
    consumes:
      consul:
        from: consul_server
        deployment: etcd

But the error message suggests it is ignoring deployment: etcd:

  - Can't resolve link 'consul_server' in instance group 'acceptance-tests' on job 'acceptance-tests' in deployment 'eats'. Please make sure the link was provided and shared.

The error message shows it is defaulting to deployment: eats rather than deployment: etcd that I specified.

Ahh, shared: true. BRB.

@drnic

This comment has been minimized.

Copy link

commented Mar 20, 2017

Thanks @Amit-PivotalLabs for this document.

For anyone else who wants to see a diff of a release that was upgraded to links; and a bosh2/cloud-config manifest + readme for deploying it, then dingotiles/etcd2-boshrelease@v95.1...v95.2 might be useful to you.

(Note: at time of writing, the eats.yml manifest errand didn't work for me; not sure if that's because the repo is a hard fork and I have to update somethings)

PR to etcd-release cloudfoundry-incubator/etcd-release#38

@drnic

This comment has been minimized.

Copy link

commented Mar 22, 2017

Perhaps include a mention of optional: true which allows manifests to use if_link("db")

consumes:
- name: db
  type: postgresql
  optional: true
@drnic

This comment has been minimized.

Copy link

commented Mar 22, 2017

Is there a guide to migrating a running pre-links deployment to a link deployment? e.g. merging explicit AZ jobs cell_z1,cell_z2 into multi-az instance groups cell?

Update: http://bosh.io/docs/migrated-from.html#migrate

instance_groups:
- name: etcd
  azs: [z1, z2]
  migrated_from:
  - {name: etcd_z1, az: z1}
  - {name: etcd_z2, az: z2}
@Amit-PivotalLabs

This comment has been minimized.

Copy link
Owner Author

commented Mar 25, 2017

Thanks @drnic, great feedback, updated accordingly. You want to use the migrated_from to take advantage of the BOSH-native notion of AZs, which is an orthogonal concern to Links. The doc you linked to captures the gist of it.

@claudio-benfatto

This comment has been minimized.

Copy link

commented Apr 4, 2017

Hello @Amit-PivotalLabs thanks for the very helpful explanation.

We are actually facing a problem while trying to integrate links with one of our bosh releases and we think that our use case may not be fully covered by the current links functionalities.
I think it could be seen as a subcase of What happens if multiple things provide the same type of link?.

The specific problem is due to the fact that we do not know in advance, (referring to the examples in this document) the number of database_instance instance groups.
At the same time we need to access them somehow dynamically from our configuration templates and specifically from the same instance group, something like the following:

instance_groups:
- name: web_server_instances
  jobs:
  - release: app
    name: web_server_process
    properties:
      greeting: "Namaste!"
    consumes:
      hindi_database_data_object_reference: {from: hindi_database}
      french_database_data_object_reference: {from: french_database}
<%
database_links = Array.new
database_links << link("hindi_database_data_object_reference")
database_links << link("french_database_data_object_reference")
%>

database_addresses={
<% database_links.each_with_index do |database, i| %>
  "<%= link("database").instances[0].address %>"<%= ',' if i < (database_links.size - 1) %>
<% end %>
}

but this ofc doesn't work very elegantly, for example, if we decided to add an italian_database_data_object_reference: {from: italian_database}, we would be forced to modify all the template files accordingly.

Would be really great to have something of this kind:

<%
database_links = link_by_type("database_data_class")
%>

database_addresses={
<% database_links.each_with_index do |database, i| %>
  "<%= link("database").instances[0].address %>"<%= ',' if i < (database_links.size - 1) %>
<% end %>
}

I have the feeling that we are missing something or trying to bend links functionalities a bit too much. Is there a better approach in order to achieve the same result?

Thanks a lot
Claudio

@Amit-PivotalLabs

This comment has been minimized.

Copy link
Owner Author

commented Apr 6, 2017

Hey @claudio-benfatto, I migrated your question here, since it'll be hard to have a threaded conversation on this Gist's comments.

@ishustava

This comment has been minimized.

Copy link

commented Sep 23, 2017

@Amit-PivotalLabs,

I believe you still do need to specify deployment name both in ~/workspace/deployments/galileo_environment/my_app_future.yml and on the command line with the -d flag, otherwise BOSH CLI will error.

@pgoodwin

This comment has been minimized.

Copy link

commented Sep 26, 2017

I'm still fairly new to BOSH and had a difficult time with some parts of this. It would have been very useful to me if one of the first things explained is that links are a mechanism for making values available for template expansion and that the mechanism for users to add their own values to this context is by exporting properties defined by spec files. If they have other effects it would be good to explicitly describe those as well.
I still haven't been able to figure out the relationship between links and manifests. The examples go to some length to highlight the uninteresting parts of the manifest, but doesn't explain what is interesting about what remains. I think that what is interesting may be something that was not required by the manifest, but to my novice eye there is nothing that I would have expected to be in the manifest that is missing. What I did notice is that every property defined by the spec is still accounted for by either a default value or definition in the manifest, so links didn't seem to do anything to make writing the manifest easier. I think it would be good to add more explanatory text about what is supposed to be noticed about the manifest file and, perhaps, comments that include the code that would otherwise have had to be written could be added to the manifest to make the effect the links are having more clear.

@Alexvianet

This comment has been minimized.

Copy link

commented Apr 13, 2018

How i can link dynamic IP of the neighbor job to manifest, not to template, like we do with static_ips:

instance_groups:
  - name: web
    properties:
      key: (( jobs.postgres.networks.default.static_ips.[0] )) 
@Amit-PivotalLabs

This comment has been minimized.

Copy link
Owner Author

commented Apr 26, 2018

Hey @Alexvianet you cannot get the value of a link to show up in a manifest. Without links, if job A needs to know the IP(s) of job B, you would have to:

  • assign static IPs to the instance group that job B is in (in the manifest)
  • ensure job A's spec defines a property that allows you to provide it B's IPs (in the BOSH release for job A)
  • copy the value of the static IPs for job B into the properties section for job A in whatever instance group it's in (in the manifest)

You may be using tools like spruce or spiff that make the static IP assignment and copying not entirely manual. However BOSH can now natively do IP assignment and property "copying" for you via Global Networking and Links, respectively. So instead the workflow would be:

  • ensure job B provides a link (in the release for job B)
  • ensure job A consumes a link, and ensure the ERB templates pull the IP info out of the link (in the release for job A)
  • if the type of the provided and consumed link match, and this is all within the same deployment manifest, then "implicit linking" will take over and you won't need to do anything else
  • otherwise, make sure you're explicit about the links being provided by job B in its instance group and consumed by job A in its instance group (in the deployment manifest)
@abh1kg

This comment has been minimized.

Copy link

commented Sep 2, 2018

@Amit-PivotalLabs Firstly, a wonderful and insightful document on the usage of BOSH links.
On the topic of BOSH links, do deployment addon jobs support links as well? I have a deployment manifest with an addon job defined. The release spec of one of the component jobs of the addon release defines a provides section. In the same manifest, I have several instance groups and component jobs which need to consume the links defined in the addon job.
I end up with an error as below:

Task 7912 | 17:16:37 | Preparing deployment: Preparing deployment (00:00:02)
                     L Error: Failed to resolve links from deployment 'target-deployment'. See errors below:
  - Failed to resolve link 'my_link' with type 'my_link_class' from job 'job' in instance group 'instance_group1'. Multiple link providers found:
    - Link provider 'my_link' from job 'addon_job' in instance group 'instance_group1' in deployment 'target-deployment'
    - Link provider 'my_link' from job 'addon_job' in instance group 'instance_group2' in deployment 'target-deployment'
@Amit-PivotalLabs

This comment has been minimized.

Copy link
Owner Author

commented Oct 4, 2018

Hey @abh1kg thanks for your praise, I'm glad you found this useful!

Unfortunately Gist comments don't result in notifications so I'm only seeing this now, apologies. I think the core BOSH team would be able to better answer your question. I've created a GitHub issue with your question here on your behalf.

Cheers!

@abh1kg

This comment has been minimized.

Copy link

commented Oct 4, 2018

Thanks again @Amit-PivotalLabs. I'll follow the GitHub issue

@lordcf

This comment has been minimized.

Copy link

commented Apr 23, 2019

Hi,
I am getting error while deploying harbor with Bosh release
Error: Link property admin_password_for_smoketest in template harbor is not defined in release spec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.