Basic no-op cloudfoundry service gateway

.gitignore

*.gem
.bundle
Gemfile.lock
pkg/*
vendor/
*~
#*
dev.yml

README.md

The project is a basic no-op cloudfoundry service gateway. You can provision and bind to it, and the app you bind to will get some credentials in VCAP_SERVICES, but otherwise it doesn't do anything.

Steps to Register with the Cloud Controller

Provide a config file

To register with the cloud controller you need to provide a config file (the default has some values in it, but won't have the right values for your environment). Then you can try and launch with, for instance

$ bin/gateway -c config/dev.yml

Note that the services base code will require /var/vcap/sys/run/LOCK to be writable. This is fixed in the dsyer fork so that the lock file location can be overridden with an environment variable:

$ LOCK_FILE=/tmp/LOCK bin/gateway -c config/dev.yml

NATS Registration

NATS registration happens before contacting the Cloud Controller so if you have problems connecting you are hosed, but you can disable it by not providing an mbus entry in the local config.

Make the Cloud Controller aware of our offering

The cloud controller has to be expecting us as well, so you need this in cloud_controller.yml the first time you run the gateway (but not subsequently):

builtin_services:
 test: 0xdeadbeef

In a BOSH deployment you can do this by adding a snippet to the manifest and then doing a bosh deploy:

external_service_tokens:
  test: 0xdeadbeef

(Except there's a bug in the cloud_controller job where the template doesn't expand the external service tokens into a hash before iterating on them.)

Make sure the gateway is not registered as "core"

The cloud controller will register the service, but you need it to be registered wit the "core" provider, so don't specify that property in the gateway config file.

Port Numbers

The Cloud Controller database doesn't seem to get updated with the new port if you change the gateweay, and the default is to pick an ephemeral port. So it's best to fix the port in the gateway YML config.

Ignore: False Steps with Marketplace Gateway

The marketplace features don't quite seem to work with the old cloud_controller. It's alternative way to get a service registered in the cloud_controller database, where the provider of the service is not "core".

Make the Cloud Controller aware of our offering

The cloud controller has to be expecting us as well, so you need this in cloud_controller.yml the first time you run the gateway (but not subsequently):

service_proxy:
 token: [0xdeadbeef]

where the token array contains the token value in the gateway config. In a BOSH deployment you can do this by adding a snippet to the manifest and then doing a bosh deploy:

marketplace_gateway:
  tokens: [0xdeadbeef]

Allow the Cloud Controller to callback to us over HTTP

You need to launch the gateway from a machine that can be contacted by the cloud controller, either by automatically discovering its IP, or by explicitly adding an ip_route to the gateway config.

Make sure the gateway is not registered as "core"

The cloud controller will register the service, but it will screw it up and register it with provider=core unless you explicitly give it a provider (not equal to "core"). If it does that it can't find it again because it always replaces "core" with nil before querying for the service. Here's a test:

$ jcurl -H "Content-Type: application/json" -H "X-VCAP-Service-Token: 0xdeadbeef" api.cf84.dev.las01.vcsops.com/services/v1/offerings/test-1.0/test

Note the further awkwardness of having to specify a content type for a GET (if you don't you get a 400 response).

VMC Problems

VMC will not send the "provider" field in the service creation call. You can do it with curl:

$ jjcurl -v -H "Authorization: bearer $TOKEN" api.cf84.dev.las01.vcsops.com/services -X POST -d '{"type":"generic","vendor":"test","version":"1.0","tier":"free","name":"test","provider":"test"}'

or you can try and stick with core services, which might mean hacking the database.

app.rb

# require 'sinatra'
$LOAD_PATH.unshift File.join(File.dirname(__FILE__), 'lib')
require 'service/gateway'

Test::Gateway.new.start

gateway

#!/usr/bin/env ruby 
# -*- mode: ruby -*- 
# Copyright (c) 2009-2011 VMware, Inc.

ENV["BUNDLE_GEMFILE"] ||= File.expand_path("../../Gemfile", __FILE__)
require 'bundler/setup'
require 'vcap_services_base'

$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', 'lib')
require 'service/gateway'
Test::Gateway.new.start

gateway.rb

require 'vcap_services_base'

$LOAD_PATH.unshift File.join(File.dirname(__FILE__), '..', 'lib')
require 'service/provisioner'

class Test::Gateway < VCAP::Services::Base::Gateway

  def provisioner_class
    Test::Provisioner
  end

  def default_config_file
    config_base_dir = ENV["CLOUD_FOUNDRY_CONFIG_PATH"] || File.join(File.dirname(__FILE__), '..', '..', 'config')
    File.join(config_base_dir, 'test_gateway.yml')
  end

end

gateway_spec.rb

#--
# Cloud Foundry 2012.02.03 Beta
# Copyright (c) [2009-2012] VMware, Inc. All Rights Reserved.
#
# This product is licensed to you under the Apache License, Version 2.0 (the "License").
# You may not use this product except in compliance with the License.
#
# This product includes a number of subcomponents with
# separate copyright notices and license terms. Your use of these
# subcomponents is subject to the terms and conditions of the
# subcomponent's license, as noted in the LICENSE file.
#++

require 'spec_helper'

module Test

  describe Gateway do

    it "should have a default config file" do
      Gateway.new().default_config_file.should_not be_nil
    end

  end

end

Gemfile

source "http://rubygems.org"

gem 'eventmachine', :git => 'git://github.com/cloudfoundry/eventmachine.git', :branch => 'release-0.12.11-cf'
gem 'vcap_common', :require => ['vcap/common', 'vcap/component'], :git => 'git://github.com/cloudfoundry/vcap-common.git', :ref => 'fd6b6d91'
gem 'vcap_logging', :require => ['vcap/logging'], :git => 'git://github.com/cloudfoundry/common.git', :ref => 'b96ec1192'
gem 'vcap_services_base', :git => 'git://github.com/dsyer/vcap-services-base.git', :ref => 'debugging'
gem 'warden-client', :require => ['warden/client'], :git => 'git://github.com/cloudfoundry/warden.git', :ref => '21f9a32ab50'
gem 'warden-protocol', :require => ['warden/protocol'], :git => 'git://github.com/cloudfoundry/warden.git', :ref => '21f9a32ab50'
gem 'cf-uaa-client', :git => 'git://github.com/cloudfoundry/uaa.git', :ref => 'master'

# Specify your gem's dependencies in test_service.gemspec
gemspec

provisioner.rb

#--
# Cloud Foundry 2012.02.03 Beta
# Copyright (c) [2009-2012] VMware, Inc. All Rights Reserved.
#
# This product is licensed to you under the Apache License, Version 2.0 (the "License").
# You may not use this product except in compliance with the License.
#
# This product includes a number of subcomponents with
# separate copyright notices and license terms. Your use of these
# subcomponents is subject to the terms and conditions of the
# subcomponent's license, as noted in the LICENSE file.
#++

module Test
end

class Test::Provisioner < VCAP::Services::Base::Provisioner

  def service_name
    "Test"
  end

  def provision_service(request, prov_handle=nil, &blk)

    @logger.debug("[#{service_description}] Attempting to provision instance (request=#{request.extract})")

    name = UUIDTools::UUID.random_create.to_s
    plan = request.plan || "free"
    version = request.version

    prov_req = ProvisionRequest.new
    prov_req.plan = request.plan
    prov_req.version = version
    # use old credentials to provision a service if provided.
    prov_req.credentials = prov_handle["credentials"] if prov_handle

    credentials = gen_credentials(name)
    svc = {
      :configuration => prov_req.dup,
      :service_id => name,
      :credentials => credentials
    }
    @logger.debug("Provisioned #{svc.inspect}")
    @prov_svcs[svc[:service_id]] = svc

    blk.call(success(svc))

  rescue => e
    @logger.warn("Exception at provision_service #{e}")
    blk.call(internal_fail)

  end

  def unprovision_service(instance_id, &blk)

    @logger.debug("[#{service_description}] Attempting to unprovision instance (instance id=#{instance_id}")
    svc = @prov_svcs[instance_id]
    raise ServiceError.new(ServiceError::NOT_FOUND, "instance_id #{instance_id}") if svc == nil
    blk.call(success())
    bindings = find_all_bindings(instance_id)
    @prov_svcs.delete(instance_id)
    bindings.each do |b|
      @prov_svcs.delete(b[:service_id])
    end

  rescue => e
    @logger.warn("Exception at unprovision_service #{e}")
    blk.call(internal_fail)

  end

  def bind_instance(instance_id, binding_options, bind_handle=nil, &blk)

    @logger.debug("[#{service_description}] Attempting to bind to service #{instance_id}")
    svc = @prov_svcs[instance_id]
    raise ServiceError.new(ServiceError::NOT_FOUND, "instance_id #{instance_id}") if svc == nil

    service_id = nil
    if bind_handle
      service_id = bind_handle["service_id"]
    else
      service_id = UUIDTools::UUID.random_create.to_s
    end

    # Save binding-options in :data section of configuration
    config = svc[:configuration].nil? ? {} : svc[:configuration].clone
    config['data'] ||= {}
    config['data']['binding_options'] = binding_options
    res = {
      :service_id => service_id,
      :configuration => config,
      :credentials => svc[:credentials]
    }
    @logger.debug("[#{service_description}] Binded: #{res.inspect}")
    @prov_svcs[res[:service_id]] = res
    blk.call(success(res))

  rescue => e
    @logger.warn("Exception at bind_instance #{e}")
    blk.call(internal_fail)

  end

  def unbind_instance(instance_id, handle_id, binding_options, &blk)

    @logger.debug("[#{service_description}] Attempting to unbind to service #{instance_id}")

    svc = @prov_svcs[instance_id]
    raise ServiceError.new(ServiceError::NOT_FOUND, "instance_id #{instance_id}") if svc == nil

    handle = @prov_svcs[handle_id]
    raise ServiceError.new(ServiceError::NOT_FOUND, "handle_id #{handle_id}") if handle.nil?

    @prov_svcs.delete(handle_id)
    blk.call(success())

  end

  def gen_credentials(name)
    credentials = {
      "internal" => {
        "name" => name
      }
    }
  end

end

provisioner_spec.rb

#--
# Cloud Foundry 2012.02.03 Beta
# Copyright (c) [2009-2012] VMware, Inc. All Rights Reserved.
#
# This product is licensed to you under the Apache License, Version 2.0 (the "License").
# You may not use this product except in compliance with the License.
#
# This product includes a number of subcomponents with
# separate copyright notices and license terms. Your use of these
# subcomponents is subject to the terms and conditions of the
# subcomponent's license, as noted in the LICENSE file.
#++

require 'spec_helper'

module Test

  describe Provisioner do

    include SpecHelper

    before :all do
      EM.run do
        @provisioner = Provisioner.new(:service=>service_config, :logger=>logger)
        EM.stop
      end
    end

    subject { @provisioner }

    def service_config
      @service_config ||= YAML.load_file(Gateway.new().default_config_file)
      @service_config.delete(:mbus)
    end
    
    it "should have a service name" do
      @provisioner.service_name.should_not be_nil
    end

    it "should provision a service" do
      request = VCAP::Services::Api::GatewayProvisionRequest.new(:label=>"test-1.0", :name=>"test", :plan=>"free", :email=>"foo@bar.com", :version=>"1.0")
      @provisioner.provision_service(request) do |svc|
        svc["success"].should be_true
        svc["response"][:credentials].should_not be_nil
      end
    end

  end

end

Rakefile

require "bundler/gem_tasks"
require "rspec/core/rake_task"

RSpec::Core::RakeTask.new("test") do |test|
  test.rspec_opts = ["--format", "documentation", "--colour"]
  test.pattern = "**/*_spec.rb"
end

service.rb

require "service/version"
require "service/gateway"
require "service/provisioner"

module Test
end

spec_helper.rb

#--
# Cloud Foundry 2012.02.03 Beta
# Copyright (c) [2009-2012] VMware, Inc. All Rights Reserved.
#
# This product is licensed to you under the Apache License, Version 2.0 (the "License").
# You may not use this product except in compliance with the License.
#
# This product includes a number of subcomponents with
# separate copyright notices and license terms. Your use of these
# subcomponents is subject to the terms and conditions of the
# subcomponent's license, as noted in the LICENSE file.
#++

require 'rspec'
require 'service'
require "logger"

module SpecHelper
  def logger
    @logger ||= Logger.new(STDOUT)
  end
end

test_gateway.yml

---
cloud_controller_uri: api.vcap.me
service:
  name: test
  version: "1.0"
  description: 'Test service'
  plans: ['free']
  default_plan: 'free'
  tags: ['test']
  timeout: 60
  supported_versions: ["1.0"]
  version_aliases:
    current: "1.0"
index: 0
mbus: nats://nats:nats@vcap:4222
logging:
  file: /tmp/test_gateway.log
  level: debug
pid: /tmp/test_service.pid
token: 0xdeadbeef

test_service.gemspec

# -*- encoding: utf-8 -*-
$:.push File.expand_path("../lib", __FILE__)
require "service/version"

Gem::Specification.new do |s|
  s.name        = "test_service"
  s.version     = Test::VERSION
  s.authors     = ["Dave Syer"]
  s.email       = ["dsyer@vmware.com"]
  s.homepage    = ""
  s.summary     = %q{TODO: Write a gem summary}
  s.description = %q{TODO: Write a gem description}

  s.rubyforge_project = "test_service"

  s.files         = `git ls-files`.split("\n")
  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
  s.require_paths = ["lib"]

  # specify any dependencies here; for example:
  s.add_development_dependency "rspec"
  s.add_runtime_dependency "vcap_common"
  s.add_runtime_dependency 'vcap_logging'
  s.add_runtime_dependency "vcap_services_base"
  s.add_runtime_dependency "cf-uaa-client"
end

version.rb

module Test
  VERSION = "0.0.1"
end