Skip to content

Instantly share code, notes, and snippets.

@groknaut

groknaut/chef errors

Created February 24, 2012 22:21
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save groknaut/1904175 to your computer and use it in GitHub Desktop.
Save groknaut/1904175 to your computer and use it in GitHub Desktop.
Download ZIP
odd errors after restoring couchdb onto new chef server
Raw
chef errors
in the first try, this config is in client.rb:
chef_server_url "https://chef.venus.spergacula.com"
chef.venus.spergacula.com is a CNAME to cheftain04.
root@cheftain04:~# chef-client -l debug -L /var/log/chef/client.log
[snip]
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing cookbook_file[/etc/sudoers] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing cookbook_file[/etc/sudoers] action create (users::user-file-dist line 6)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing directory[/home/builder] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing directory[/home/builder] action create (users::user-file-dist line 16)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing cookbook_file[/home/builder/.gitconfig] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing cookbook_file[/home/builder/.gitconfig] action create (users::user-file-dist line 22)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing directory[/home/billeh] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing directory[/home/billeh] action create (users::user-file-dist line 37)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing cookbook_file[/home/billeh/.gitconfig] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing cookbook_file[/home/billeh/.gitconfig] action create (users::user-file-dist line 43)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Processing cookbook_file[/home/billeh/.ssh/known_hosts] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] INFO: Processing cookbook_file[/home/billeh/.ssh/known_hosts] action create (users::user-file-dist line 49)
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Signing the request as cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: String to sign: 'Method:GET
Hashed Path:gEQfzh2aC15IzI6AQKCf3HPgz+w=
X-Ops-Content-Hash:2jmj7l5rSw0yVb/vlWAYkK/YBwk=
X-Ops-Timestamp:2012-02-24T21:52:54Z
X-Ops-UserId:cheftain04.venus.spergacula.com'
Header hash: {"X-Ops-Content-Hash"=>"2jmj7l5rSw0yVb/vlWAYkK/YBwk=", "X-Ops-Authorization-1"=>"kjxvoJlULlFYBXHpoDST8WX2WWV+a9NsADIUVqqHlj+LHqYEt7UBnXd237Lt", "X-Ops-Userid"=>"cheftain04.venus.spergacula.com", "X-Ops-Sign"=>"version=1.0", "X-Ops-Authorization-2"=>"ACiW/8ngaCtULq5JS+uSIi5wTrdYRpWQQJ/VUjgQpnLNQf/qYCutmJlBH3NF", "X-Ops-Authorization-3"=>"J/DxMlQjFnlsumUe1NdOq5RWuBPfFZch8ATVZWTUCFiIPkRKN1Rgcvpp5erR", "X-Ops-Authorization-4"=>"EvmqTfg40P6XCHVWKiQ3OyJ2tBQ8AvGyvnMDCRwUiVXJXJgRUnmCtIVjppZT", "X-Ops-Authorization-5"=>"0HquNrAByFSucMzeeNU7g9A9MDr8oThkbhYvdq3Jgky3Y8NTPtTYhHlnRNjd", "X-Ops-Authorization-6"=>"zjnqBs9hyLqsJkr0gWdXs05sNuQDG2c4ipaEUgohfQ==", "X-Ops-Timestamp"=>"2012-02-24T21:52:54Z"}
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Sending HTTP Request via GET to chef.venus.spergacula.com:443/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0
[Fri, 24 Feb 2012 21:52:54 +0000] DEBUG: Streaming download from https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0 to tempfile /tmp/chef-rest20120224-3188-okzovs-0
[Fri, 24 Feb 2012 21:52:54 +0000] ERROR: Server returned error for https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0, retrying 1/5 in 3s
[Fri, 24 Feb 2012 21:52:57 +0000] DEBUG: Streaming download from https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0 to tempfile /tmp/chef-rest20120224-3188-h4rtbr-0
[Fri, 24 Feb 2012 21:52:57 +0000] ERROR: Server returned error for https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0, retrying 2/5 in 5s
[Fri, 24 Feb 2012 21:53:02 +0000] DEBUG: Streaming download from https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0 to tempfile /tmp/chef-rest20120224-3188-iywyo5-0
[Fri, 24 Feb 2012 21:53:02 +0000] ERROR: Server returned error for https://chef.venus.spergacula.com/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0, retrying 3/5 in 11s
^C[Fri, 24 Feb 2012 21:53:03 +0000] FATAL: SIGINT received, stopping
[Fri, 24 Feb 2012 21:53:03 +0000] ERROR: Running exception handlers
[Fri, 24 Feb 2012 21:53:04 +0000] FATAL: Saving node information to /var/cache/chef/failed-run-data.json
[Fri, 24 Feb 2012 21:53:04 +0000] ERROR: Exception handlers complete
[Fri, 24 Feb 2012 21:53:04 +0000] DEBUG: Re-raising exception: SystemExit - exit
/usr/lib/ruby/vendor_ruby/chef/application.rb:143:in `exit'
/usr/lib/ruby/vendor_ruby/chef/application.rb:143:in `fatal!'
/usr/lib/ruby/vendor_ruby/chef/application.rb:39:in `initialize'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:319:in `call'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:319:in `sleep'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:319:in `retriable_rest_request'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:259:in `streaming_request'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:112:in `get_rest'
/usr/lib/ruby/vendor_ruby/chef/cookbook/remote_file_vendor.rb:63:in `get_filename'
/usr/lib/ruby/vendor_ruby/chef/cookbook_version.rb:659:in `preferred_filename_on_disk_location'
/usr/lib/ruby/vendor_ruby/chef/provider/cookbook_file.rb:64:in `file_cache_location'
/usr/lib/ruby/vendor_ruby/chef/provider/cookbook_file.rb:36:in `action_create'
/usr/lib/ruby/vendor_ruby/chef/resource.rb:440:in `send'
/usr/lib/ruby/vendor_ruby/chef/resource.rb:440:in `run_action'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:45:in `run_action'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `converge'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `each'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `converge'
/usr/lib/ruby/vendor_ruby/chef/resource_collection.rb:94:in `execute_each_resource'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:85:in `step'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/usr/lib/ruby/vendor_ruby/chef/resource_collection.rb:92:in `execute_each_resource'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:76:in `converge'
/usr/lib/ruby/vendor_ruby/chef/client.rb:312:in `converge'
/usr/lib/ruby/vendor_ruby/chef/client.rb:160:in `run'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:239:in `run_application'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:229:in `loop'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:229:in `run_application'
/usr/lib/ruby/vendor_ruby/chef/application.rb:67:in `run'
/usr/bin/chef-client:25
nginx proxies 443 to chef at 4000. logs at this time:
==> error.log <==
2012/02/24 21:53:02 [error] 869#0: *353 upstream prematurely closed connection while reading response header from upstream, client: 10.248.0.89, server: chef.venus.spergacula.com, request: "GET /cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0 HTTP/1.1", upstream: "http://127.0.0.1:4000/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0", host: "chef.venus.spergacula.com:443"
==> access.log <==
10.248.0.89 - - [24/Feb/2012:21:53:02 +0000] "GET /cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0 HTTP/1.1" 502 173 "-" "Chef Client/0.10.8 (ruby-1.8.7-p302; ohai-0.6.10; x86_64-linux; +http://opscode.com)"
in the second try, this config is in client.rb, for shizzles. and the
errror is different.
chef_server_url "http://localhost:4000"
[snip]
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing cookbook_file[/etc/sudoers] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing cookbook_file[/etc/sudoers] action create (users::user-file-dist line 6)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing directory[/home/builder] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing directory[/home/builder] action create (users::user-file-dist line 16)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing cookbook_file[/home/builder/.gitconfig] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing cookbook_file[/home/builder/.gitconfig] action create (users::user-file-dist line 22)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing directory[/home/billeh] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing directory[/home/billeh] action create (users::user-file-dist line 37)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing cookbook_file[/home/billeh/.gitconfig] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing cookbook_file[/home/billeh/.gitconfig] action create (users::user-file-dist line 43)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Processing cookbook_file[/home/billeh/.ssh/known_hosts] on cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] INFO: Processing cookbook_file[/home/billeh/.ssh/known_hosts] action create (users::user-file-dist line 49)
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Signing the request as cheftain04.venus.spergacula.com
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: String to sign: 'Method:GET
Hashed Path:gEQfzh2aC15IzI6AQKCf3HPgz+w=
X-Ops-Content-Hash:2jmj7l5rSw0yVb/vlWAYkK/YBwk=
X-Ops-Timestamp:2012-02-24T21:55:03Z
X-Ops-UserId:cheftain04.venus.spergacula.com'
Header hash: {"X-Ops-Content-Hash"=>"2jmj7l5rSw0yVb/vlWAYkK/YBwk=", "X-Ops-Authorization-1"=>"aSCFY4+m47xSwe2MSjC+pMALoQVAf3As6fOE2+A55Bv6PU/TdAIn6x13nd5d", "X-Ops-Userid"=>"cheftain04.venus.spergacula.com", "X-Ops-Sign"=>"version=1.0", "X-Ops-Authorization-2"=>"QTkrxr6kMcMRigVUI9RZwvfh1Lld2u+BTqqEEPEAO1rETU2RomFdXE1mJ/e8", "X-Ops-Authorization-3"=>"dMhcOzy48/Yb27X+m92n/PfOd8pI4WpuGM4z/IIqWNHsZc/FFxwmGBcYDZQZ", "X-Ops-Authorization-4"=>"O9F+PByr66KLRILoibmIlGE+D9N7mM0oZXa7o/CpTGVUUwRvOYwcXHINLLGi", "X-Ops-Authorization-5"=>"VyyC2ppEJ5eFBOZs4AFEzDdaj1D1b+1EViXK5eBfKxuXzymN3s3zp76AZWIY", "X-Ops-Authorization-6"=>"S+uqamW/65VZ/NbYQW/lvc9N9aIlrjP0Jhs4r+rwdQ==", "X-Ops-Timestamp"=>"2012-02-24T21:55:03Z"}
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Sending HTTP Request via GET to localhost:4000/cookbooks/users/1.0.0/files/bb27ab0eeedaa8f2da0d3f8fc3039df0
[Fri, 24 Feb 2012 21:55:03 +0000] ERROR: cookbook_file[/home/billeh/.ssh/known_hosts] (users::user-file-dist line 49) has had an error
[Fri, 24 Feb 2012 21:55:03 +0000] ERROR: cookbook_file[/home/billeh/.ssh/known_hosts] (/var/cache/chef/cookbooks/users/recipes/user-file-dist.rb:49:in `from_file') had an error:
cookbook_file[/home/billeh/.ssh/known_hosts] (users::user-file-dist line 49) had an error: EOFError: end of file reached
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:52:in `sysread'
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:52:in `rbuf_fill'
/usr/lib/ruby/1.8/timeout.rb:67:in `timeout'
/usr/lib/ruby/1.8/timeout.rb:101:in `timeout'
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:51:in `rbuf_fill'
/usr/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
/usr/lib/ruby/1.8/net/protocol.rb:126:in `readline'
/usr/lib/ruby/1.8/net/http.rb:2028:in `read_status_line'
/usr/lib/ruby/1.8/net/http.rb:2017:in `read_new'
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:146:in `request'
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:131:in `request'
/usr/lib/ruby/1.8/net/http.rb:543:in `start'
/var/lib/gems/1.8/gems/right_http_connection-1.3.0/lib/net_fix.rb:129:in `request'
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:84:in `call'
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:99:in `hide_net_http_bug'
/usr/lib/ruby/vendor_ruby/chef/rest/rest_request.rb:83:in `call'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:261:in `streaming_request'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:296:in `retriable_rest_request'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:259:in `streaming_request'
/usr/lib/ruby/vendor_ruby/chef/rest.rb:112:in `get_rest'
/usr/lib/ruby/vendor_ruby/chef/cookbook/remote_file_vendor.rb:63:in `get_filename'
/usr/lib/ruby/vendor_ruby/chef/cookbook_version.rb:659:in `preferred_filename_on_disk_location'
/usr/lib/ruby/vendor_ruby/chef/provider/cookbook_file.rb:64:in `file_cache_location'
/usr/lib/ruby/vendor_ruby/chef/provider/cookbook_file.rb:36:in `action_create'
/usr/lib/ruby/vendor_ruby/chef/resource.rb:440:in `send'
/usr/lib/ruby/vendor_ruby/chef/resource.rb:440:in `run_action'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:45:in `run_action'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `converge'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `each'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:81:in `converge'
/usr/lib/ruby/vendor_ruby/chef/resource_collection.rb:94:in `execute_each_resource'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:116:in `call'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:85:in `step'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/usr/lib/ruby/vendor_ruby/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/usr/lib/ruby/vendor_ruby/chef/resource_collection.rb:92:in `execute_each_resource'
/usr/lib/ruby/vendor_ruby/chef/runner.rb:76:in `converge'
/usr/lib/ruby/vendor_ruby/chef/client.rb:312:in `converge'
/usr/lib/ruby/vendor_ruby/chef/client.rb:160:in `run'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:239:in `run_application'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:229:in `loop'
/usr/lib/ruby/vendor_ruby/chef/application/client.rb:229:in `run_application'
/usr/lib/ruby/vendor_ruby/chef/application.rb:67:in `run'
/usr/bin/chef-client:25
[Fri, 24 Feb 2012 21:55:03 +0000] ERROR: Running exception handlers
[Fri, 24 Feb 2012 21:55:03 +0000] FATAL: Saving node information to /var/cache/chef/failed-run-data.json
[Fri, 24 Feb 2012 21:55:03 +0000] ERROR: Exception handlers complete
[Fri, 24 Feb 2012 21:55:03 +0000] DEBUG: Re-raising exception: EOFError - cookbook_file[/home/billeh/.ssh/known_hosts] (users::user-file-dist line 49) had an error: EOFError: end of file reached
[snip more of the same]
here is the portion of users::user-file-dist complained about above:
43 cookbook_file "/home/billeh/.gitconfig" do
44 source "billeh.gitconfig"
45 mode 0644
46 owner "billeh"
47 group "billeh"
48 end
49 cookbook_file "/home/billeh/.ssh/known_hosts" do
50 source "billeh.github.known_hosts"
51 mode 0644
52 owner "billeh"
53 group "billeh"
54 end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment