Failure with second "bar" instance

chef-client run

> chef-client -c .chef/client.rb -o dev02::web_server -l info
[2014-05-28T14:12:25-07:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 11.12.4
[2014-05-28T14:12:25-07:00] INFO: *** Chef 11.12.4 ***
[2014-05-28T14:12:25-07:00] INFO: Chef-client pid: 5297
[2014-05-28T14:12:30-07:00] WARN: Run List override has been provided.
[2014-05-28T14:12:30-07:00] WARN: Original Run List: []
[2014-05-28T14:12:30-07:00] WARN: Overridden Run List: [recipe[dev02::web_server]]
[2014-05-28T14:12:30-07:00] INFO: Run List is [recipe[dev02::web_server]]
[2014-05-28T14:12:30-07:00] INFO: Run List expands to [dev02::web_server]
[2014-05-28T14:12:30-07:00] INFO: Starting Chef Run for rberger_Cymek
[2014-05-28T14:12:30-07:00] INFO: Running start handlers
[2014-05-28T14:12:30-07:00] INFO: Start handlers complete.
resolving cookbooks for run list: ["dev02::web_server"]
[2014-05-28T14:12:31-07:00] INFO: Loading cookbooks [dev02@0.1.0]
Synchronizing Cookbooks:
[2014-05-28T14:12:31-07:00] INFO: Storing updated cookbooks/dev02/recipes/web_server.rb in the cache.
  - dev02
Compiling Cookbooks...
Converging 2 resources
Recipe: dev02::web_server
  * machine_batch[default] action converge[2014-05-28T14:12:32-07:00] INFO: Processing machine_batch[default] action converge (dev02::web_server line 20)
[2014-05-28T14:12:32-07:00] INFO: HTTP Request Returned 404 Object Not Found: error
[2014-05-28T14:12:33-07:00] INFO: Using AWS profile rfspot ...

    - [bar] creating machine bar on fog:AWS:115448733555
    - [bar]   key_name: "dev02-east"
    - [bar]   tags: {"Name"=>"bar", "BootstrapId"=>"https://api.opscode.com/organizations/portfolia/nodes/bar", "BootstrapHost"=>"Cymek.local", "BootstrapUser"=>"rberger"}
    - [bar]   name: "bar"
    - [bar] machine bar created as i-7372e420 on fog:AWS:115448733555[2014-05-28T14:12:39-07:00] INFO: Processing chef_node[bar] action create (basic_chef_client::block line 46)
[2014-05-28T14:12:39-07:00] INFO: HTTP Request Returned 404 Object Not Found: error

    - create node bar at https://api.opscode.com/organizations/portfolia
    -   add normal.tags = nil
    -   add normal.metal = {"location"=>{"driver_url"=>"fog:AWS:115448733555", "driver_version"=>"0.5.beta.2", "server_id"=>"i-7372e420", "creator"=>"user/rberger@ibd.com", "allocated_at"=>"2014-05-28 21:12:39 UTC", "key_name"=>"dev02-east"}}
    - [bar] waiting for bar (i-7372e420 on fog:AWS:115448733555) to be ready ...
    - [bar] bar is now ready[2014-05-28T14:12:59-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79

    - [bar] waiting for bar (i-7372e420 on fog:AWS:115448733555) to be connectable (transport up and running) ...[2014-05-28T14:13:11-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:12-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:14-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:16-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:17-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:19-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:21-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:23-07:00] INFO: Executing sudo pwd on ubuntu@54.205.53.79
[2014-05-28T14:13:31-07:00] INFO: Completed pwd on ubuntu@54.205.53.79: exit status 0

    - [bar] bar is now connectable[2014-05-28T14:13:31-07:00] INFO: Processing chef_node[bar] action create (basic_chef_client::block line 46)
[2014-05-28T14:13:33-07:00] INFO: Processing private_key[in_memory] action create (basic_chef_client::block line 81)

    - [bar] create rsa private key none (2048 bits)[2014-05-28T14:13:33-07:00] INFO: Executing sudo ls -d /etc/chef/client.pem on ubuntu@54.205.53.79
[2014-05-28T14:13:33-07:00] INFO: Completed ls -d /etc/chef/client.pem on ubuntu@54.205.53.79: exit status 2
[2014-05-28T14:13:33-07:00] INFO: Executing sudo ls -d /etc/chef on ubuntu@54.205.53.79
[2014-05-28T14:13:33-07:00] INFO: Completed ls -d /etc/chef on ubuntu@54.205.53.79: exit status 2
[2014-05-28T14:13:33-07:00] INFO: Executing sudo mkdir -p /etc/chef on ubuntu@54.205.53.79
[2014-05-28T14:13:33-07:00] INFO: Completed mkdir -p /etc/chef on ubuntu@54.205.53.79: exit status 0

    - [bar] create directory /etc/chef on bar[2014-05-28T14:13:33-07:00] INFO: Executing sudo mkdir -p /etc/chef on ubuntu@54.205.53.79
[2014-05-28T14:13:34-07:00] INFO: Completed mkdir -p /etc/chef on ubuntu@54.205.53.79: exit status 0
[2014-05-28T14:13:34-07:00] INFO: Executing sudo mv /tmp/client.pem.625436040 /etc/chef/client.pem on ubuntu@54.205.53.79
[2014-05-28T14:13:34-07:00] INFO: Completed mv /tmp/client.pem.625436040 /etc/chef/client.pem on ubuntu@54.205.53.79: exit status 0

    - [bar] write file /etc/chef/client.pem on bar[2014-05-28T14:13:34-07:00] INFO: Processing chef_client[bar] action create (basic_chef_client::block line 133)
[2014-05-28T14:13:35-07:00] INFO: HTTP Request Returned 404 Object Not Found: error
[2014-05-28T14:13:35-07:00] INFO: HTTP Request Returned 403 Forbidden: error
[2014-05-28T14:13:35-07:00] INFO: Running queued delayed notifications before re-raising exception

================================================================================
Error executing action `converge` on resource 'machine_batch[default]'
================================================================================


Net::HTTPServerException
------------------------
chef_client[bar] (basic_chef_client::block line 133) had an error: Net::HTTPServerException: 403 "Forbidden"


Resource Declaration:
---------------------
# In /Users/rberger/work/rfspot/rfspot-metal/.chef/cache/cookbooks/dev02/recipes/web_server.rb

 20: machine "bar"



Compiled Resource:
------------------
# Declared in /Users/rberger/work/rfspot/rfspot-metal/.chef/cache/cookbooks/dev02/recipes/web_server.rb:20:in `from_file'

machine_batch("default") do
  action :converge
  retries 0
  retry_delay 2
  guard_interpreter :default
  machines [<machine[bar] @name: "bar" @noop: nil @before: nil @params: {} @provider: nil @allowed_actions: [:nothing, :allocate, :ready, :setup, :converge, :converge_only, :destroy, :stop, :converge] @action: :converge @updated: false @updated_by_last_action: false @supports: {} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "/Users/rberger/work/rfspot/rfspot-metal/.chef/cache/cookbooks/dev02/recipes/web_server.rb:20:in `from_file'" @guard_interpreter: :default @elapsed_time: 0 @resource_name: :machine @chef_environment: nil @chef_server: {:chef_server_url=>"https://api.opscode.com/organizations/portfolia", :options=>{:client_name=>"rberger_Cymek", :signing_key_filename=>"/Users/rberger/.chef/keys/portfolia/rberger_Cymek.pem"}} @driver: "fog:AWS:rfspot" @machine_options: {:bootstrap_options=>{:key_name=>"dev02-east"}} @cookbook_name: "dev02" @recipe_name: "web_server" @complete: nil @raw_json: nil @run_list: nil @attributes: nil @allow_overwrite_keys: nil @source_key: nil @source_key_path: nil @source_key_pass_phrase: nil @private_key_options: nil @ohai_hints: nil @public_key_path: nil @public_key_format: nil @admin: nil @validator: nil>]
  chef_server {:chef_server_url=>"https://api.opscode.com/organizations/portfolia", :options=>{:client_name=>"rberger_Cymek", :signing_key_filename=>"/Users/rberger/.chef/keys/portfolia/rberger_Cymek.pem"}}
  cookbook_name "dev02"
  recipe_name "web_server"
end



[2014-05-28T14:13:35-07:00] INFO: Running queued delayed notifications before re-raising exception

Running handlers:
[2014-05-28T14:13:35-07:00] ERROR: Running exception handlers
Running handlers complete

[2014-05-28T14:13:35-07:00] ERROR: Exception handlers complete
[2014-05-28T14:13:35-07:00] FATAL: Stacktrace dumped to /Users/rberger/work/rfspot/rfspot-metal/.chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 69.976351 seconds
[2014-05-28T14:13:35-07:00] INFO: Sending resource update report (run-id: 03638a22-3aa6-495c-9355-56c985c377e4)
[2014-05-28T14:13:36-07:00] ERROR: machine_batch[default] (dev02::web_server line 20) had an error: Net::HTTPServerException: chef_client[bar] (basic_chef_client::block line 133) had an error: Net::HTTPServerException: 403 "Forbidden"
[2014-05-28T14:13:36-07:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)