Skip to content

Instantly share code, notes, and snippets.

@barkerd427

barkerd427/gist:72b4795c9021cb8863ea

Created January 16, 2015 14:06
Show Gist options
  • Save barkerd427/72b4795c9021cb8863ea to your computer and use it in GitHub Desktop.
Save barkerd427/72b4795c9021cb8863ea to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
2015-01-15 14:22:28,454 INFO process-manager [139879117629392] [process_manager.py:65] Launching ['/var/lib/cattle/pyagent/cattle/process_watcher.sh']
2015-01-15 14:22:28,457 INFO process-manager [139879117629392] [process_manager.py:68] Launched ['/var/lib/cattle/pyagent/cattle/process_watcher.sh'] as pid 472
2015-01-15 14:22:28,466 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: host-api from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,466 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: libvirt_config_drive from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,467 INFO libvirt [139879117629392] [__init__.py:17] Failed to find libvirt python
2015-01-15 14:22:28,468 INFO libvirt [139879117629392] [__init__.py:41] Disabling libvirt
2015-01-15 14:22:28,471 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: libvirt_directory_pool from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,471 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: docker from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,668 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: console-agent from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,669 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: libvirt_vnc_console from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,670 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: cadvisor from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,671 INFO agent [139879117629392] [__init__.py:15] Loading Plugin: libvirt_qemu_volume from /var/lib/cattle/pyagent/cattle/plugins
2015-01-15 14:22:28,671 INFO agent [139879117629392] [main.py:89] API URL http://ipfacecobld28:8080/v1
2015-01-15 14:22:28,673 INFO agent [139879117629392] [main.py:95] Subscribing to ['storage.image.activate', 'storage.volume.activate', 'storage.volume.deactivate', 'storage.volume.remove', 'ping', 'config.update', 'delegate.request', 'compute.instance.activate', 'compute.instance.deactivate']
2015-01-15 14:22:28,673 INFO process-manager [139879117629392] [process_manager.py:65] Launching ['host-api', '-cadvisor-url', 'http://127.0.0.1:9344', '-logtostderr=true', '-ip', '0.0.0.0', '-port', '9345']
2015-01-15 14:22:28,677 INFO process-manager [139879117629392] [process_manager.py:68] Launched ['host-api', '-cadvisor-url', 'http://127.0.0.1:9344', '-logtostderr=true', '-ip', '0.0.0.0', '-port', '9345'] as pid 477
2015-01-15 14:22:28,678 INFO process-manager [139879117629392] [process_manager.py:65] Launching ['nodejs', '/var/lib/cattle/console-agent/agent.js', '-d', 'http://unix:/var/run/docker.sock:', '-k', '/var/lib/cattle/etc/cattle/api.crt', '-p', '9346']
2015-01-15 14:22:28,682 INFO process-manager [139879117629392] [process_manager.py:68] Launched ['nodejs', '/var/lib/cattle/console-agent/agent.js', '-d', 'http://unix:/var/run/docker.sock:', '-k', '/var/lib/cattle/etc/cattle/api.crt', '-p', '9346'] as pid 479
2015-01-15 14:22:28,683 INFO process-manager [139879117629392] [process_manager.py:65] Launching ['nsenter', '--mount=/host/proc/1/ns/mnt', '--', '/var/lib/cattle/pyagent/cattle/plugins/cadvisor/cadvisor', '-logtostderr=true', '-ip', '127.0.0.1', '-port', '9344']
2015-01-15 14:22:28,686 INFO process-manager [139879117629392] [process_manager.py:68] Launched ['nsenter', '--mount=/host/proc/1/ns/mnt', '--', '/var/lib/cattle/pyagent/cattle/plugins/cadvisor/cadvisor', '-logtostderr=true', '-ip', '127.0.0.1', '-port', '9344'] as pid 480
2015-01-15 14:22:28,691 INFO requests.packages.urllib3.connectionpool [139879105752752] [connectionpool.py:188] Starting new HTTP connection (1): ipfacecobld28
2015-01-15 14:22:29,650 INFO requests.packages.urllib3.connectionpool [139879102762832] [connectionpool.py:188] Starting new HTTP connection (1): ipfacecobld28
2015-01-15 14:23:04,660 INFO agent [139879105405008] [utils.py:351] Request: {"id":"3cbe7656-7938-4971-9f9d-134c445a7c90","name":"storage.image.activate","replyTo":"reply.103936617522734639","resourceId":"2","resourceType":"imageStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"imageStoragePoolMap":{"description":null,"data":{"fields":{}},"id":2,"imageId":1,"created":1421331784604,"kind":"imageStoragePoolMap","removeTime":null,"removed":null,"uuid":"49f6794f-2cd8-4171-8c23-9e6f3f9e22bd","storagePoolId":1,"name":null,"state":"activating","type":"imageStoragePoolMap","storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"},"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}}},"time":1421331784641,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"28","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->imagestoragepoolmap.activate->(ImageStoragePoolMapActivate)","logicName":"ImageStoragePoolMapActivate","prettyResource":"instance:1->imageStoragePoolMap:2","processName":"imagestoragepoolmap.activate","topResourceType":"instance","resouceId":"2","resouceType":"imageStoragePoolMap","prettyProcess":"instance.start->imagestoragepoolmap.activate","topProcessName":"instance.start","topResourceId":"1","processUuid":"16bee8e0-666a-4b16-9cbc-e0efcfdab461"}}
2015-01-15 14:23:04,661 INFO agent [139879105405008] [utils.py:351] worker1 : Starting request 3cbe7656-7938-4971-9f9d-134c445a7c90 for storage.image.activate
2015-01-15 14:23:04,780 INFO agent [139879105405008] [utils.py:351] Response: {"name": "reply.103936617522734639", "resourceType": "imageStoragePoolMap", "resourceId": "2", "previousNames": ["storage.image.activate"], "time": 1421331784000, "previousIds": ["3cbe7656-7938-4971-9f9d-134c445a7c90"], "data": {"+data": {"dockerImage": {"Created": 1420098664, "VirtualSize": 91749442, "ParentId": "9dbe459d586488dff75b0b3867b6482ed3839e7a02040babbf77359c763a46f0", "RepoTags": ["nginx:1", "nginx:1.7", "nginx:latest"], "Id": "1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064", "Size": 0}}}, "id": "82dfbceb-c6cb-4a9b-b169-df90b953b548"} [0.00974082946777] seconds
2015-01-15 14:23:04,780 INFO agent [139879105405008] [utils.py:351] worker1 : Done request 3cbe7656-7938-4971-9f9d-134c445a7c90 for storage.image.activate [0.119534015656] seconds
2015-01-15 14:23:04,849 INFO agent [139879105406448] [utils.py:351] Request: {"id":"f0e8fe0d-bf6e-4dee-becb-6b1102b5fb7a","name":"storage.volume.activate","replyTo":"reply.8683357558469968090","resourceId":"1","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":1,"created":1421331784388,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"14e5408f-c503-4737-8360-dc9dd705ea1d","storagePoolId":1,"volumeId":1,"name":null,"state":"activating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":null,"uri":null,"id":1,"deviceNumber":0,"imageId":1,"instanceId":1,"offeringId":null,"zoneId":1,"accountId":1,"created":1421331784194,"kind":"volume","removeTime":null,"removed":null,"uuid":"797791ac-766d-4702-a14f-c39e26cc4231","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"activating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[1],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningMessage":"create storage","transitioningProgress":10}},"token":"LwExl8qDFvw13iJjpsCjCiAzOM5Yt7pSiITX5Z4DZfNFQMl5PbHJVifqqSNL559PTAeMw3DLyl66UxyuAJQ","id":1,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421331784076,"kind":"container","removeTime":null,"removed":null,"uuid":"6d1de42d-1d81-426c-94cb-9feea1d19990","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421331784828,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"28","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->volumestoragepoolmap.activate->(VolumeStoragePoolMapActivate)","logicName":"VolumeStoragePoolMapActivate","prettyResource":"instance:1->volumeStoragePoolMap:1","processName":"volumestoragepoolmap.activate","topResourceType":"instance","resouceId":"1","resouceType":"volumeStoragePoolMap","prettyProcess":"instance.start->volumestoragepoolmap.activate","topProcessName":"instance.start","topResourceId":"1","processUuid":"16bee8e0-666a-4b16-9cbc-e0efcfdab461"}}
2015-01-15 14:23:04,849 INFO agent [139879105406448] [utils.py:351] worker10 : Starting request f0e8fe0d-bf6e-4dee-becb-6b1102b5fb7a for storage.volume.activate
2015-01-15 14:23:04,859 INFO agent [139879105406448] [utils.py:351] Response: {"name": "reply.8683357558469968090", "resourceType": "volumeStoragePoolMap", "resourceId": "1", "previousNames": ["storage.volume.activate"], "time": 1421331784000, "previousIds": ["f0e8fe0d-bf6e-4dee-becb-6b1102b5fb7a"], "data": {"volume": {"format": "docker"}}, "id": "9fe09489-fbdf-4d87-84a4-e367aaf47d79"} [0.00897216796875] seconds
2015-01-15 14:23:04,860 INFO agent [139879105406448] [utils.py:351] worker10 : Done request f0e8fe0d-bf6e-4dee-becb-6b1102b5fb7a for storage.volume.activate [0.0102100372314] seconds
2015-01-15 14:23:05,026 INFO agent [139879105405808] [utils.py:351] Request: {"id":"75996ea8-8a61-4490-938b-db5e3d6129f4","name":"compute.instance.activate","replyTo":"reply.8683357558469968090","resourceId":"1","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":1,"instanceId":1,"hostId":1,"created":1421331784382,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"0af2fb2d-7f8b-4c37-bd89-c9f95a64e94c","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[1],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningMessage":"create compute","transitioningProgress":90},"metadataAttach":true,"metadata":{"user-data":null,"meta-data":{"placement":{"availability-zone":"zone1"},"local-hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","services":{"domain":"amazonaws.com"},"ami-manifest-path":"(unknown)","hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","mac":"02:2e:ba:84:fb:b7","reservation-id":"1i1","network":{"interfaces":{"macs":{"02:2e:ba:84:fb:b7":{"local-hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","owner-id":"1i1","device-number":0,"security-groups":"","security-group-ids":"","mac":"02:2e:ba:84:fb:b7","vpc-id":"1n1","vpc-ipv4-cidr-block":null}}}},"ami-launch-index":"0","block-device-mapping":{"ami":"/dev/sda","root":"/dev/sda1"},"instance-id":"i-1i1","instance-action":"none","kernel-id":"1i1","ami-id":"1i1","instance-type":"256mb-1cpu","profile":"default-paravirtual"}}},"token":"LwExl8qDFvw13iJjpsCjCiAzOM5Yt7pSiITX5Z4DZfNFQMl5PbHJVifqqSNL559PTAeMw3DLyl66UxyuAJQ","id":1,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421331784076,"kind":"container","removeTime":null,"removed":null,"uuid":"6d1de42d-1d81-426c-94cb-9feea1d19990","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":1,"deviceNumber":0,"imageId":1,"instanceId":1,"offeringId":null,"zoneId":1,"accountId":1,"created":1421331784194,"kind":"volume","removeTime":null,"removed":null,"uuid":"797791ac-766d-4702-a14f-c39e26cc4231","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[1],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningMessage":"create compute","transitioningProgress":90},"metadataAttach":true,"metadata":{"user-data":null,"meta-data":{"placement":{"availability-zone":"zone1"},"local-hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","services":{"domain":"amazonaws.com"},"ami-manifest-path":"(unknown)","hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","mac":"02:2e:ba:84:fb:b7","reservation-id":"1i1","network":{"interfaces":{"macs":{"02:2e:ba:84:fb:b7":{"local-hostname":"ip-02-2e-ba-84-fb-b7.compute.internal","owner-id":"1i1","device-number":0,"security-groups":"","security-group-ids":"","mac":"02:2e:ba:84:fb:b7","vpc-id":"1n1","vpc-ipv4-cidr-block":null}}}},"ami-launch-index":"0","block-device-mapping":{"ami":"/dev/sda","root":"/dev/sda1"},"instance-id":"i-1i1","instance-action":"none","kernel-id":"1i1","ami-id":"1i1","instance-type":"256mb-1cpu","profile":"default-paravirtual"}}},"token":"LwExl8qDFvw13iJjpsCjCiAzOM5Yt7pSiITX5Z4DZfNFQMl5PbHJVifqqSNL559PTAeMw3DLyl66UxyuAJQ","id":1,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421331784076,"kind":"container","removeTime":null,"removed":null,"uuid":"6d1de42d-1d81-426c-94cb-9feea1d19990","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":1,"deviceNumber":0,"instanceId":1,"macAddress":"02:2e:ba:84:fb:b7","networkId":1,"subnetId":null,"vnetId":null,"accountId":1,"created":1421331784237,"kind":"nic","removeTime":null,"removed":null,"uuid":"5e215b1f-514d-454f-bb7d-3f4b5fc07129","name":null,"state":"active","type":"nic","ipAddresses":[],"network":{"domain":null,"description":null,"data":{"fields":{"macPrefix":"02:2e:ba"}},"id":1,"isPublic":true,"accountId":2,"created":1421331724223,"kind":"network","removeTime":null,"removed":null,"uuid":"unmanaged","name":"Unmanaged Network","state":"active","type":"network","networkServiceProviders":[],"networkServices":[{"description":null,"data":{"fields":{"configDrive":true}},"id":8,"networkId":1,"accountId":2,"created":1421331724973,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"unmanaged-docker0-metadata-service","networkServiceProviderId":null,"name":"Meta data service for unmanaged docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999999,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421331784999,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"28","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:1->instanceHostMap:1","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"1","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"1","processUuid":"16bee8e0-666a-4b16-9cbc-e0efcfdab461"}}
2015-01-15 14:23:05,026 INFO agent [139879105405808] [utils.py:351] worker6 : Starting request 75996ea8-8a61-4490-938b-db5e3d6129f4 for compute.instance.activate
2015-01-15 14:23:05,046 INFO docker [139879105405808] [compute.py:293] Creating docker container [6d1de42d-1d81-426c-94cb-9feea1d19990] from config {'tty': False, 'stdin_open': False, 'name': u'6d1de42d-1d81-426c-94cb-9feea1d19990', 'memswap_limit': None, 'environment': {}, 'detach': True, 'hostname': None}
2015-01-15 14:23:05,561 INFO docker [139879105405808] [compute.py:311] Starting docker container [6d1de42d-1d81-426c-94cb-9feea1d19990] docker id [479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589] {'publish_all_ports': True, 'links': {}, 'cap_add': [], 'lxc_conf': {}, 'dns_search': [], 'dns': [], 'cap_drop': [], 'privileged': False}
2015-01-15 14:23:06,228 INFO agent [139879105405808] [utils.py:351] Response: {"name": "reply.8683357558469968090", "resourceType": "instanceHostMap", "resourceId": "1", "previousNames": ["compute.instance.activate"], "time": 1421331786000, "previousIds": ["75996ea8-8a61-4490-938b-db5e3d6129f4"], "data": {"instance": {"+data": {"dockerInspect": {"AppArmorProfile": "", "Id": "479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589", "HostsPath": "/var/lib/docker/containers/479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589/hosts", "Created": "2015-01-15T14:23:05.057887518Z", "Image": "1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064", "Args": ["-g", "daemon off;"], "Driver": "devicemapper", "HostConfig": {"CapDrop": null, "ContainerIDFile": "", "NetworkMode": "", "Links": null, "LxcConf": [], "PortBindings": null, "SecurityOpt": null, "CapAdd": null, "Binds": null, "RestartPolicy": {"MaximumRetryCount": 0, "Name": ""}, "Devices": null, "Dns": [], "ExtraHosts": null, "DnsSearch": null, "Privileged": false, "VolumesFrom": null, "PublishAllPorts": true}, "VolumesRW": {"/var/cache/nginx": true}, "State": {"Pid": 12038, "Paused": false, "Running": true, "FinishedAt": "0001-01-01T00:00:00Z", "Restarting": false, "StartedAt": "2015-01-15T14:23:06.190147616Z", "ExitCode": 0}, "ExecDriver": "native-0.2", "ResolvConfPath": "/var/lib/docker/containers/479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589/resolv.conf", "Volumes": {"/var/cache/nginx": "/var/lib/docker/vfs/dir/edfaa63f56511d3266ba3eab3d1692d971f13df7860a27109bffa223f88c0255"}, "Path": "nginx", "HostnamePath": "/var/lib/docker/containers/479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589/hostname", "ProcessLabel": "", "Config": {"Volumes": {"/var/cache/nginx": {}}, "Hostname": "479add0d4e83", "Entrypoint": null, "Env": ["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "NGINX_VERSION=1.7.9-1~wheezy"], "Memory": 0, "OnBuild": null, "OpenStdin": false, "Cpuset": "", "User": "", "AttachStderr": false, "AttachStdout": false, "NetworkDisabled": false, "WorkingDir": "", "Cmd": ["nginx", "-g", "daemon off;"], "StdinOnce": false, "PortSpecs": null, "AttachStdin": false, "MemorySwap": 0, "Tty": false, "CpuShares": 0, "Domainname": "", "Image": "nginx:latest", "ExposedPorts": {"443/tcp": {}, "80/tcp": {}}}, "MountLabel": "", "NetworkSettings": {"MacAddress": "02:42:ac:11:00:0e", "Bridge": "docker0", "PortMapping": null, "IPPrefixLen": 16, "IPAddress": "172.17.0.14", "Gateway": "172.17.42.1", "Ports": {"443/tcp": [{"HostIp": "0.0.0.0", "HostPort": "49153"}], "80/tcp": [{"HostIp": "0.0.0.0", "HostPort": "49154"}]}}, "Name": "/6d1de42d-1d81-426c-94cb-9feea1d19990"}, "dockerContainer": {"Status": "Up Less than a second", "Created": 1421331785, "Image": "nginx:1", "Id": "479add0d4e8324f30acc799c105b0486bfa9481050b418e9cd5f9b71a93ec589", "Command": "nginx -g 'daemon off;'", "Names": ["/6d1de42d-1d81-426c-94cb-9feea1d19990"], "Ports": [{"PublicPort": 49153, "IP": "0.0.0.0", "Type": "tcp", "PrivatePort": 443}, {"PublicPort": 49154, "IP": "0.0.0.0", "Type": "tcp", "PrivatePort": 80}]}, "+fields": {"dockerHostIp": "172.17.42.1", "dockerPorts": {"443/tcp": "49153", "80/tcp": "49154"}, "dockerIp": "172.17.0.14"}}}}, "id": "c141e616-b760-4cd1-8ea0-2b267a359adc"} [0.0130369663239] seconds
2015-01-15 14:23:06,229 INFO agent [139879105405808] [utils.py:351] worker6 : Done request 75996ea8-8a61-4490-938b-db5e3d6129f4 for compute.instance.activate [1.20252799988] seconds
2015-01-15 14:23:06,472 INFO agent [139879105406288] [utils.py:351] Request: {"id":"59e0ef1f-9886-46eb-951f-be52cf7b1442","name":"storage.volume.activate","replyTo":"reply.103936617522734639","resourceId":"2","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":2,"created":1421331786408,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"e4da0300-9cff-4969-8daa-7f89206cdd83","storagePoolId":1,"volumeId":2,"name":null,"state":"activating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{"isHostPath":false}},"format":null,"uri":"file:///var/lib/docker/vfs/dir/edfaa63f56511d3266ba3eab3d1692d971f13df7860a27109bffa223f88c0255","id":2,"deviceNumber":-1,"imageId":null,"instanceId":null,"offeringId":null,"zoneId":1,"accountId":1,"created":1421331786403,"kind":"volume","removeTime":null,"removed":null,"uuid":"4bad0215-4b4c-4b75-9b59-bf361ce3240c","attachedState":"inactive","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"inactive","type":"volume","offering":null,"instance":null,"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":null},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421331786468,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"28","logicName":"VolumeStoragePoolMapActivate","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(DockerPostInstanceHostMapActivate)->volumestoragepoolmap.activate->(VolumeStoragePoolMapActivate)","processName":"volumestoragepoolmap.activate","prettyResource":"instance:1->volumeStoragePoolMap:2","topResourceType":"instance","resouceId":"2","resouceType":"volumeStoragePoolMap","topProcessName":"instance.start","prettyProcess":"instance.start->volumestoragepoolmap.activate","topResourceId":"1","processUuid":"16bee8e0-666a-4b16-9cbc-e0efcfdab461"}}
2015-01-15 14:23:06,472 INFO agent [139879105406288] [utils.py:351] worker9 : Starting request 59e0ef1f-9886-46eb-951f-be52cf7b1442 for storage.volume.activate
2015-01-15 14:23:06,477 INFO agent [139879105406288] [utils.py:351] Response: {"name": "reply.103936617522734639", "resourceType": "volumeStoragePoolMap", "resourceId": "2", "previousNames": ["storage.volume.activate"], "time": 1421331786000, "previousIds": ["59e0ef1f-9886-46eb-951f-be52cf7b1442"], "data": {"volume": {"format": "docker"}}, "id": "48818c80-d801-4b3a-8bf1-fcb2b4b25a45"} [0.00432395935059] seconds
2015-01-15 14:23:06,477 INFO agent [139879105406288] [utils.py:351] worker9 : Done request 59e0ef1f-9886-46eb-951f-be52cf7b1442 for storage.volume.activate [0.00498414039612] seconds
2015-01-15 14:29:37,188 INFO agent [139879105407408] [utils.py:351] Request: {"id":"415087a1-e9d6-44d6-a1c5-e9d97d98f270","name":"storage.volume.activate","replyTo":"reply.7900026675093608484","resourceId":"3","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":3,"created":1421332174980,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"ecdd497b-d128-4adf-b51c-67cfd30a97a6","storagePoolId":1,"volumeId":3,"name":null,"state":"activating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":null,"uri":null,"id":3,"deviceNumber":0,"imageId":1,"instanceId":2,"offeringId":null,"zoneId":1,"accountId":1,"created":1421332174860,"kind":"volume","removeTime":null,"removed":null,"uuid":"388dcd2e-4288-4699-b4f0-7611587d0876","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"activating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[2],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningMessage":"create storage","transitioningProgress":10}},"token":"jTsh3Wi51gLhj2npel92uwATYo5MHcQFsvdWlBu6Khhl7r4VabrXWBc3doJMKAw5Q30LmLCNg0Q3flgGT9ZIg","id":2,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421332174789,"kind":"container","removeTime":null,"removed":null,"uuid":"24bc0c8e-3886-48d4-8008-9f6df502e6a0","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332177178,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"29","logicName":"VolumeStoragePoolMapActivate","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->volumestoragepoolmap.activate->(VolumeStoragePoolMapActivate)","processName":"volumestoragepoolmap.activate","prettyResource":"instance:2->volumeStoragePoolMap:3","topResourceType":"instance","resouceId":"3","resouceType":"volumeStoragePoolMap","topProcessName":"instance.start","prettyProcess":"instance.start->volumestoragepoolmap.activate","topResourceId":"2","processUuid":"3f221bd7-f037-4d65-aae2-6287553512c7"}}
2015-01-15 14:29:37,188 INFO agent [139879105407408] [utils.py:351] worker16 : Starting request 415087a1-e9d6-44d6-a1c5-e9d97d98f270 for storage.volume.activate
2015-01-15 14:29:37,199 INFO agent [139879105407408] [utils.py:351] Response: {"name": "reply.7900026675093608484", "resourceType": "volumeStoragePoolMap", "resourceId": "3", "previousNames": ["storage.volume.activate"], "time": 1421332177000, "previousIds": ["415087a1-e9d6-44d6-a1c5-e9d97d98f270"], "data": {"volume": {"format": "docker"}}, "id": "5edafad3-c0e9-4a5e-9e2f-4195a068eeca"} [0.00999093055725] seconds
2015-01-15 14:29:37,199 INFO agent [139879105407408] [utils.py:351] worker16 : Done request 415087a1-e9d6-44d6-a1c5-e9d97d98f270 for storage.volume.activate [0.011253118515] seconds
2015-01-15 14:29:37,485 INFO agent [139879105407088] [utils.py:351] Request: {"id":"e9f39153-fa66-41fb-8261-06069644fec2","name":"storage.image.activate","replyTo":"reply.7900026675093608484","resourceId":"4","resourceType":"imageStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"imageStoragePoolMap":{"description":null,"data":{"fields":{}},"id":4,"imageId":2,"created":1421332177465,"kind":"imageStoragePoolMap","removeTime":null,"removed":null,"uuid":"d239c4a0-3c2f-4692-a716-6c61bcb91767","storagePoolId":1,"name":null,"state":"activating","type":"imageStoragePoolMap","storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"},"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}},"time":1421332177478,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"34","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->imagestoragepoolmap.activate->(ImageStoragePoolMapActivate)","logicName":"ImageStoragePoolMapActivate","prettyResource":"instance:3->imageStoragePoolMap:4","processName":"imagestoragepoolmap.activate","topResourceType":"instance","resouceId":"4","resouceType":"imageStoragePoolMap","prettyProcess":"instance.start->imagestoragepoolmap.activate","topProcessName":"instance.start","topResourceId":"3","processUuid":"a251ff4a-488b-4507-bfbe-3c1a1113d76d"}}
2015-01-15 14:29:37,486 INFO agent [139879105407088] [utils.py:351] worker14 : Starting request e9f39153-fa66-41fb-8261-06069644fec2 for storage.image.activate
2015-01-15 14:29:37,559 INFO agent [139879105407088] [utils.py:351] Response: {"name": "reply.7900026675093608484", "resourceType": "imageStoragePoolMap", "resourceId": "4", "previousNames": ["storage.image.activate"], "time": 1421332177000, "previousIds": ["e9f39153-fa66-41fb-8261-06069644fec2"], "data": {"+data": {"dockerImage": {"Created": 1415407212, "VirtualSize": 501760064, "ParentId": "f75667cf7100cba2c04ca649d1e483a2ae958c47e2a67a1c89f7dfa65c63052d", "RepoTags": ["rancher/agent-instance:v0.1.0"], "Id": "545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b", "Size": 126962203}}}, "id": "59289d89-bf7c-4acb-a845-683821709019"} [0.00494289398193] seconds
2015-01-15 14:29:37,559 INFO agent [139879105407088] [utils.py:351] worker14 : Done request e9f39153-fa66-41fb-8261-06069644fec2 for storage.image.activate [0.0732290744781] seconds
2015-01-15 14:29:37,579 INFO agent [139879105405488] [utils.py:351] Request: {"id":"3df63c01-d442-4af7-a4e4-deaf08b82022","name":"storage.volume.activate","replyTo":"reply.7900026675093608484","resourceId":"4","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":4,"created":1421332177185,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"fb6c1820-e4f5-4d45-923e-e62acc936d79","storagePoolId":1,"volumeId":4,"name":null,"state":"activating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":null,"uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"activating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create storage","transitioningProgress":10,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332177575,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"34","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->volumestoragepoolmap.activate->(VolumeStoragePoolMapActivate)","logicName":"VolumeStoragePoolMapActivate","prettyResource":"instance:3->volumeStoragePoolMap:4","processName":"volumestoragepoolmap.activate","topResourceType":"instance","resouceId":"4","resouceType":"volumeStoragePoolMap","prettyProcess":"instance.start->volumestoragepoolmap.activate","topProcessName":"instance.start","topResourceId":"3","processUuid":"a251ff4a-488b-4507-bfbe-3c1a1113d76d"}}
2015-01-15 14:29:37,580 INFO agent [139879105405488] [utils.py:351] worker4 : Starting request 3df63c01-d442-4af7-a4e4-deaf08b82022 for storage.volume.activate
2015-01-15 14:29:37,586 INFO agent [139879105405488] [utils.py:351] Response: {"name": "reply.7900026675093608484", "resourceType": "volumeStoragePoolMap", "resourceId": "4", "previousNames": ["storage.volume.activate"], "time": 1421332177000, "previousIds": ["3df63c01-d442-4af7-a4e4-deaf08b82022"], "data": {"volume": {"format": "docker"}}, "id": "dff0b9d6-0440-4661-9fe9-359ab38900fa"} [0.00546598434448] seconds
2015-01-15 14:29:37,586 INFO agent [139879105405488] [utils.py:351] worker4 : Done request 3df63c01-d442-4af7-a4e4-deaf08b82022 for storage.volume.activate [0.00633502006531] seconds
2015-01-15 14:29:37,673 INFO agent [139879105406608] [utils.py:351] Request: {"id":"c38f5125-564b-4727-9ed7-1c0aa76f1f04","name":"compute.instance.activate","replyTo":"reply.7900026675093608484","resourceId":"3","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":3,"instanceId":3,"hostId":1,"created":1421332177174,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"1dfaf32c-2e8b-45b5-8e7f-af3485bb1787","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":3,"deviceNumber":0,"instanceId":3,"macAddress":"02:77:36:fd:a2:4c","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332177059,"kind":"nic","removeTime":null,"removed":null,"uuid":"b99f9a20-1be5-4ddd-8223-657e11d07804","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.30.37","data":{"fields":{}},"role":"primary","id":4,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332177357,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"29794e3a-9573-4b27-8afb-dd86c638b3ce","hostname":null,"name":"10.42.30.37","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332177660,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"34","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:3->instanceHostMap:3","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"3","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"3","processUuid":"a251ff4a-488b-4507-bfbe-3c1a1113d76d"}}
2015-01-15 14:29:37,673 INFO agent [139879105406608] [utils.py:351] worker11 : Starting request c38f5125-564b-4727-9ed7-1c0aa76f1f04 for compute.instance.activate
2015-01-15 14:29:37,701 INFO docker [139879105406608] [compute.py:293] Creating docker container [4c296f1c-e41f-4031-8f26-b10c6fb75302] from config {'environment': {'CATTLE_CONFIG_URL': 'http://ipfacecobld28:8080/v1', u'CATTLE_AGENT_INSTANCE_AUTH': u'Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=='}, 'detach': True, 'name': u'4c296f1c-e41f-4031-8f26-b10c6fb75302', 'hostname': None}
2015-01-15 14:29:38,483 INFO docker [139879105406608] [compute.py:311] Starting docker container [4c296f1c-e41f-4031-8f26-b10c6fb75302] docker id [6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:29:39,829 ERROR cattle [139879105406608] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '16740', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
43: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
44: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:0f brd ff:ff:ff:ff:ff:ff
43: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
44: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:0f brd ff:ff:ff:ff:ff:ff
inet 172.17.0.15/16 scope global eth0
inet6 fe80::42:acff:fe11:f/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.30.37/16 to eth0
43: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
44: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:0f brd ff:ff:ff:ff:ff:ff
43: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
44: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 02:42:ac:11:00:0f brd ff:ff:ff:ff:ff:ff
inet 172.17.0.15/16 scope global eth0
inet 10.42.30.37/16 scope global eth0
inet6 fe80::42:acff:fe11:f/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:fd:a2:4c
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '16740', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:29:39,832 INFO agent [139879105406608] [utils.py:351] worker11 : Done request c38f5125-564b-4727-9ed7-1c0aa76f1f04 for compute.instance.activate [2.15833902359] seconds
2015-01-15 14:29:39,832 ERROR agent [139879105406608] [event.py:116] 75ce1abe-6caf-4141-8201-988f91a393fe : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '16740', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:29:39,847 INFO agent [139879105406608] [utils.py:351] Response: {"name": "reply.7900026675093608484", "resourceType": "instanceHostMap", "resourceId": "3", "id": "7aa32b0a-afda-4bb2-b514-18848519a570", "transitioningInternalMessage": "75ce1abe-6caf-4141-8201-988f91a393fe : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '16740', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332179000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["c38f5125-564b-4727-9ed7-1c0aa76f1f04"]} [0.0119500160217] seconds
2015-01-15 14:29:47,822 INFO agent [139879105407248] [utils.py:351] Request: {"id":"a7970975-b49c-4b66-a3e9-cffb026a31e7","name":"compute.instance.activate","replyTo":"reply.956103037962096539","resourceId":"3","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":3,"instanceId":3,"hostId":1,"created":1421332177174,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"1dfaf32c-2e8b-45b5-8e7f-af3485bb1787","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":3,"deviceNumber":0,"instanceId":3,"macAddress":"02:77:36:fd:a2:4c","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332177059,"kind":"nic","removeTime":null,"removed":null,"uuid":"b99f9a20-1be5-4ddd-8223-657e11d07804","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.30.37","data":{"fields":{}},"role":"primary","id":4,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332177357,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"29794e3a-9573-4b27-8afb-dd86c638b3ce","hostname":null,"name":"10.42.30.37","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332187811,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"34","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:3->instanceHostMap:3","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"3","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"3","processUuid":"be2466ac-c806-4130-a9e5-5ad3467376ac"}}
2015-01-15 14:29:47,823 INFO agent [139879105407248] [utils.py:351] worker15 : Starting request a7970975-b49c-4b66-a3e9-cffb026a31e7 for compute.instance.activate
2015-01-15 14:29:47,854 INFO docker [139879105407248] [compute.py:311] Starting docker container [4c296f1c-e41f-4031-8f26-b10c6fb75302] docker id [6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:29:48,142 ERROR cattle [139879105407248] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17021', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
46: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
47: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:10 brd ff:ff:ff:ff:ff:ff
46: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
47: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:10 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.16/16 scope global eth0
inet6 fe80::42:acff:fe11:10/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.30.37/16 to eth0
46: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
47: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:10 brd ff:ff:ff:ff:ff:ff
46: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
47: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:10 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.16/16 scope global eth0
inet 10.42.30.37/16 scope global eth0
inet6 fe80::42:acff:fe11:10/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:fd:a2:4c
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17021', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:29:48,145 INFO agent [139879105407248] [utils.py:351] worker15 : Done request a7970975-b49c-4b66-a3e9-cffb026a31e7 for compute.instance.activate [0.322136163712] seconds
2015-01-15 14:29:48,146 ERROR agent [139879105407248] [event.py:116] 3235380f-fc6c-4baa-a233-993ef9d60fe9 : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17021', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:29:48,160 INFO agent [139879105407248] [utils.py:351] Response: {"name": "reply.956103037962096539", "resourceType": "instanceHostMap", "resourceId": "3", "id": "cd25aec6-d605-48c4-8552-d34e4f8d0331", "transitioningInternalMessage": "3235380f-fc6c-4baa-a233-993ef9d60fe9 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17021', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332188000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["a7970975-b49c-4b66-a3e9-cffb026a31e7"]} [0.0125410556793] seconds
2015-01-15 14:30:02,793 INFO agent [139879105405008] [utils.py:351] Request: {"id":"d0aa3e8c-f54e-43ab-a635-f1bdd229cc9c","name":"compute.instance.activate","replyTo":"reply.2081647492522876205","resourceId":"3","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":3,"instanceId":3,"hostId":1,"created":1421332177174,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"1dfaf32c-2e8b-45b5-8e7f-af3485bb1787","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":3,"deviceNumber":0,"instanceId":3,"macAddress":"02:77:36:fd:a2:4c","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332177059,"kind":"nic","removeTime":null,"removed":null,"uuid":"b99f9a20-1be5-4ddd-8223-657e11d07804","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.30.37","data":{"fields":{}},"role":"primary","id":4,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332177357,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"29794e3a-9573-4b27-8afb-dd86c638b3ce","hostname":null,"name":"10.42.30.37","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332202775,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"34","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:3->instanceHostMap:3","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"3","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"3","processUuid":"d123a908-f286-4bc0-8579-78056e70d64f"}}
2015-01-15 14:30:02,793 INFO agent [139879105405008] [utils.py:351] worker1 : Starting request d0aa3e8c-f54e-43ab-a635-f1bdd229cc9c for compute.instance.activate
2015-01-15 14:30:02,834 INFO docker [139879105405008] [compute.py:311] Starting docker container [4c296f1c-e41f-4031-8f26-b10c6fb75302] docker id [6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:30:03,145 ERROR cattle [139879105405008] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
49: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
50: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:11 brd ff:ff:ff:ff:ff:ff
49: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
50: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:11 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.17/16 scope global eth0
inet6 fe80::42:acff:fe11:11/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.30.37/16 to eth0
49: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
50: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:11 brd ff:ff:ff:ff:ff:ff
49: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
50: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:11 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.17/16 scope global eth0
inet 10.42.30.37/16 scope global eth0
inet6 fe80::42:acff:fe11:11/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:fd:a2:4c
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:30:03,148 INFO agent [139879105405008] [utils.py:351] worker1 : Done request d0aa3e8c-f54e-43ab-a635-f1bdd229cc9c for compute.instance.activate [0.354393959045] seconds
2015-01-15 14:30:03,149 ERROR agent [139879105405008] [event.py:116] 75f8534e-ba2e-4a9b-893e-3c9c11d6a4e1 : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:30:03,158 INFO agent [139879105405008] [utils.py:351] Response: {"name": "reply.2081647492522876205", "resourceType": "instanceHostMap", "resourceId": "3", "id": "83162256-ae12-4b4e-a64a-52e21b8bb548", "transitioningInternalMessage": "75f8534e-ba2e-4a9b-893e-3c9c11d6a4e1 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332203000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["d0aa3e8c-f54e-43ab-a635-f1bdd229cc9c"]} [0.00792598724365] seconds
2015-01-15 14:30:03,256 INFO agent [139879105405488] [utils.py:351] Request: {"id":"bf7b7610-edd3-40c7-a81b-05693e577ddd","name":"compute.instance.deactivate","replyTo":"reply.6899025339177352175","resourceId":"3","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":3,"instanceId":3,"hostId":1,"created":1421332177174,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"1dfaf32c-2e8b-45b5-8e7f-af3485bb1787","name":null,"state":"deactivating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : 75f8534e-ba2e-4a9b-893e-3c9c11d6a4e1 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : 75f8534e-ba2e-4a9b-893e-3c9c11d6a4e1 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":3,"deviceNumber":0,"instanceId":3,"macAddress":"02:77:36:fd:a2:4c","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332177059,"kind":"nic","removeTime":null,"removed":null,"uuid":"b99f9a20-1be5-4ddd-8223-657e11d07804","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.30.37","data":{"fields":{}},"role":"primary","id":4,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332177357,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"29794e3a-9573-4b27-8afb-dd86c638b3ce","hostname":null,"name":"10.42.30.37","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332203236,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"36","logicName":"InstanceHostMapDeactivate","logicPath":"instance.remove->(InstanceRemove)->instancehostmap.deactivate->(InstanceHostMapDeactivate)","processName":"instancehostmap.deactivate","prettyResource":"instance:3->instanceHostMap:3","topResourceType":"instance","resouceId":"3","resouceType":"instanceHostMap","topProcessName":"instance.remove","prettyProcess":"instance.remove->instancehostmap.deactivate","topResourceId":"3","processUuid":"0535e902-4f5f-4fd0-833f-cbcf69171c97"}}
2015-01-15 14:30:03,256 INFO agent [139879105405488] [utils.py:351] worker4 : Starting request bf7b7610-edd3-40c7-a81b-05693e577ddd for compute.instance.deactivate
2015-01-15 14:30:04,438 INFO agent [139879105405488] [utils.py:351] Response: {"name": "reply.6899025339177352175", "resourceType": "instanceHostMap", "resourceId": "3", "previousNames": ["compute.instance.deactivate"], "time": 1421332204000, "previousIds": ["bf7b7610-edd3-40c7-a81b-05693e577ddd"], "data": {"instance": {"+data": {"dockerInspect": {"AppArmorProfile": "", "Id": "6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d", "HostsPath": "/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hosts", "Created": "2015-01-15T14:29:37.719254721Z", "Image": "545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b", "Args": ["init"], "Driver": "devicemapper", "HostConfig": {"CapDrop": null, "ContainerIDFile": "", "NetworkMode": "", "Links": null, "LxcConf": null, "PortBindings": {"4500/udp": [{"HostIp": "0.0.0.0", "HostPort": "4500"}], "500/udp": [{"HostIp": "0.0.0.0", "HostPort": "500"}]}, "SecurityOpt": null, "CapAdd": null, "Binds": null, "RestartPolicy": {"MaximumRetryCount": 0, "Name": ""}, "Devices": null, "Dns": null, "ExtraHosts": null, "DnsSearch": null, "Privileged": true, "VolumesFrom": null, "PublishAllPorts": false}, "VolumesRW": {}, "State": {"Pid": 0, "Paused": false, "Running": false, "FinishedAt": "2015-01-15T14:30:04.349298912Z", "Restarting": false, "StartedAt": "2015-01-15T14:30:03.086981005Z", "ExitCode": -1}, "ExecDriver": "native-0.2", "ResolvConfPath": "/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/resolv.conf", "Volumes": {}, "Path": "/etc/init.d/agent-instance-startup", "HostnamePath": "/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hostname", "ProcessLabel": "", "Config": {"Volumes": {}, "Hostname": "6af4b3e8bde4", "Entrypoint": null, "Env": ["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1", "CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==", "HOME=/", "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"], "Memory": 0, "OnBuild": null, "OpenStdin": false, "Cpuset": "", "User": "", "AttachStderr": false, "AttachStdout": false, "NetworkDisabled": false, "WorkingDir": "", "Cmd": ["/etc/init.d/agent-instance-startup", "init"], "StdinOnce": false, "PortSpecs": null, "AttachStdin": false, "MemorySwap": 0, "Tty": false, "CpuShares": 0, "Domainname": "", "Image": "rancher/agent-instance:v0.1.0", "ExposedPorts": {}}, "MountLabel": "", "NetworkSettings": {"MacAddress": "", "Bridge": "", "PortMapping": null, "IPPrefixLen": 0, "IPAddress": "", "Gateway": "", "Ports": null}, "Name": "/4c296f1c-e41f-4031-8f26-b10c6fb75302"}, "dockerContainer": {"Status": "Exited (-1) Less than a second ago", "Created": 1421332177, "Image": "rancher/agent-instance:v0.1.0", "Id": "6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d", "Command": "/etc/init.d/agent-instance-startup init", "Names": ["/4c296f1c-e41f-4031-8f26-b10c6fb75302"], "Ports": []}, "+fields": {"dockerHostIp": "172.17.42.1", "dockerPorts": {}, "dockerIp": ""}}}}, "id": "9d6922e6-4b7f-4aaf-840d-ae4c84ded7a0"} [0.0105111598969] seconds
2015-01-15 14:30:04,438 INFO agent [139879105405488] [utils.py:351] worker4 : Done request bf7b7610-edd3-40c7-a81b-05693e577ddd for compute.instance.deactivate [1.18232893944] seconds
2015-01-15 14:30:04,557 INFO agent [139879105407568] [utils.py:351] Request: {"id":"04c02322-f73c-426a-b9a4-627684559304","name":"storage.volume.deactivate","replyTo":"reply.5957613878758951766","resourceId":"4","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":4,"created":1421332177185,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"fb6c1820-e4f5-4d45-923e-e62acc936d79","storagePoolId":1,"volumeId":4,"name":null,"state":"deactivating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":null,"removed":null,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"deactivating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : 75f8534e-ba2e-4a9b-893e-3c9c11d6a4e1 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '17311', '-i', '10.42.30.37/16', '-m', u'02:77:36:fd:a2:4c', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":"","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"dockerInspect":{"AppArmorProfile":"","Id":"6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d","HostsPath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hosts","Created":"2015-01-15T14:29:37.719254721Z","Image":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","Args":["init"],"Driver":"devicemapper","HostConfig":{"CapDrop":null,"ContainerIDFile":"","NetworkMode":"","Links":null,"LxcConf":null,"PortBindings":{"4500/udp":[{"HostIp":"0.0.0.0","HostPort":"4500"}],"500/udp":[{"HostIp":"0.0.0.0","HostPort":"500"}]},"SecurityOpt":null,"CapAdd":null,"Binds":null,"RestartPolicy":{"MaximumRetryCount":0,"Name":""},"Devices":null,"Dns":null,"ExtraHosts":null,"DnsSearch":null,"Privileged":true,"VolumesFrom":null,"PublishAllPorts":false},"VolumesRW":{},"State":{"Pid":0,"Paused":false,"Running":false,"FinishedAt":"2015-01-15T14:30:04.349298912Z","Restarting":false,"StartedAt":"2015-01-15T14:30:03.086981005Z","ExitCode":-1},"ExecDriver":"native-0.2","ResolvConfPath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/resolv.conf","Volumes":{},"Path":"/etc/init.d/agent-instance-startup","HostnamePath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hostname","ProcessLabel":"","Config":{"Volumes":{},"Hostname":"6af4b3e8bde4","Entrypoint":null,"Env":["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1","CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==","HOME=/","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Memory":0,"OnBuild":null,"OpenStdin":false,"Cpuset":"","User":"","AttachStderr":false,"AttachStdout":false,"NetworkDisabled":false,"WorkingDir":"","Cmd":["/etc/init.d/agent-instance-startup","init"],"StdinOnce":false,"PortSpecs":null,"AttachStdin":false,"MemorySwap":0,"Tty":false,"CpuShares":0,"Domainname":"","Image":"rancher/agent-instance:v0.1.0","ExposedPorts":{}},"MountLabel":"","NetworkSettings":{"MacAddress":"","Bridge":"","PortMapping":null,"IPPrefixLen":0,"IPAddress":"","Gateway":"","Ports":null},"Name":"/4c296f1c-e41f-4031-8f26-b10c6fb75302"},"dockerContainer":{"Status":"Exited (-1) Less than a second ago","Created":1421332177,"Image":"rancher/agent-instance:v0.1.0","Id":"6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d","Command":"/etc/init.d/agent-instance-startup init","Names":["/4c296f1c-e41f-4031-8f26-b10c6fb75302"],"Ports":[]},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":null,"removed":null,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332204538,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"36","logicName":"VolumeStoragePoolMapDeactivate","logicPath":"instance.remove->(InstanceRemove)->volume.deactivate->(VolumeDeactivate)->volumestoragepoolmap.deactivate->(VolumeStoragePoolMapDeactivate)","processName":"volumestoragepoolmap.deactivate","prettyResource":"instance:3->volumeStoragePoolMap:4","topResourceType":"instance","resouceId":"4","resouceType":"volumeStoragePoolMap","topProcessName":"instance.remove","prettyProcess":"instance.remove->volumestoragepoolmap.deactivate","topResourceId":"3","processUuid":"0535e902-4f5f-4fd0-833f-cbcf69171c97"}}
2015-01-15 14:30:04,557 INFO agent [139879105407568] [utils.py:351] worker17 : Starting request 04c02322-f73c-426a-b9a4-627684559304 for storage.volume.deactivate
2015-01-15 14:30:04,567 INFO agent [139879105407568] [utils.py:351] Response: {"name": "reply.5957613878758951766", "resourceType": "volumeStoragePoolMap", "resourceId": "4", "previousNames": ["storage.volume.deactivate"], "time": 1421332204000, "previousIds": ["04c02322-f73c-426a-b9a4-627684559304"], "data": {"volume": {"format": "docker"}}, "id": "8f6d9b67-1ee9-438b-b26f-0cdb325409b4"} [0.00835108757019] seconds
2015-01-15 14:30:04,567 INFO agent [139879105407568] [utils.py:351] worker17 : Done request 04c02322-f73c-426a-b9a4-627684559304 for storage.volume.deactivate [0.00968098640442] seconds
2015-01-15 14:31:18,237 INFO agent [139879102762032] [utils.py:351] Request: {"id":"a3a1d249-4f2e-4220-b272-ba6f26e44092","name":"storage.volume.remove","replyTo":"reply.3886858951383430044","resourceId":"4","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":4,"created":1421332177185,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"fb6c1820-e4f5-4d45-923e-e62acc936d79","storagePoolId":1,"volumeId":4,"name":null,"state":"removing","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":4,"deviceNumber":0,"imageId":2,"instanceId":3,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332177038,"kind":"volume","removeTime":1421332264634,"removed":1421332204634,"uuid":"8feb5d80-ebc1-4e8c-a39c-bac5018d61f2","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"deactivating","name":null,"state":"purging","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":"","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"dockerInspect":{"AppArmorProfile":"","Id":"6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d","HostsPath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hosts","Created":"2015-01-15T14:29:37.719254721Z","Image":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","Args":["init"],"Driver":"devicemapper","HostConfig":{"CapDrop":null,"ContainerIDFile":"","NetworkMode":"","Links":null,"LxcConf":null,"PortBindings":{"4500/udp":[{"HostIp":"0.0.0.0","HostPort":"4500"}],"500/udp":[{"HostIp":"0.0.0.0","HostPort":"500"}]},"SecurityOpt":null,"CapAdd":null,"Binds":null,"RestartPolicy":{"MaximumRetryCount":0,"Name":""},"Devices":null,"Dns":null,"ExtraHosts":null,"DnsSearch":null,"Privileged":true,"VolumesFrom":null,"PublishAllPorts":false},"VolumesRW":{},"State":{"Pid":0,"Paused":false,"Running":false,"FinishedAt":"2015-01-15T14:30:04.349298912Z","Restarting":false,"StartedAt":"2015-01-15T14:30:03.086981005Z","ExitCode":-1},"ExecDriver":"native-0.2","ResolvConfPath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/resolv.conf","Volumes":{},"Path":"/etc/init.d/agent-instance-startup","HostnamePath":"/var/lib/docker/containers/6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d/hostname","ProcessLabel":"","Config":{"Volumes":{},"Hostname":"6af4b3e8bde4","Entrypoint":null,"Env":["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1","CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==","HOME=/","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Memory":0,"OnBuild":null,"OpenStdin":false,"Cpuset":"","User":"","AttachStderr":false,"AttachStdout":false,"NetworkDisabled":false,"WorkingDir":"","Cmd":["/etc/init.d/agent-instance-startup","init"],"StdinOnce":false,"PortSpecs":null,"AttachStdin":false,"MemorySwap":0,"Tty":false,"CpuShares":0,"Domainname":"","Image":"rancher/agent-instance:v0.1.0","ExposedPorts":{}},"MountLabel":"","NetworkSettings":{"MacAddress":"","Bridge":"","PortMapping":null,"IPPrefixLen":0,"IPAddress":"","Gateway":"","Ports":null},"Name":"/4c296f1c-e41f-4031-8f26-b10c6fb75302"},"dockerContainer":{"Status":"Exited (-1) Less than a second ago","Created":1421332177,"Image":"rancher/agent-instance:v0.1.0","Id":"6af4b3e8bde4f5ba8902a02579413c812a70fb157ded4520b6bf88c406738c5d","Command":"/etc/init.d/agent-instance-startup init","Names":["/4c296f1c-e41f-4031-8f26-b10c6fb75302"],"Ports":[]},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"CGgk2iIWu5cTQoPHt7TPsn3zShYnFaHxH4ZazvMz2LGgbFXirYrhbzKDbT9GHMKy3yJaFU6AArM7K6ODMQ","id":3,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332176949,"kind":"container","removeTime":1421332264645,"removed":1421332204645,"uuid":"4c296f1c-e41f-4031-8f26-b10c6fb75302","compute":null,"firstRunning":null,"allocationState":"inactive","hostname":null,"name":"Network Agent","state":"purging","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332278222,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"39","logicName":"VolumeStoragePoolMapRemove","logicPath":"volume.purge->(VolumePurge)->volume.deallocate->(VolumeDeallocate)->volumestoragepoolmap.remove->(VolumeStoragePoolMapRemove)","processName":"volumestoragepoolmap.remove","prettyResource":"volume:4->volumeStoragePoolMap:4","topResourceType":"volume","resouceId":"4","resouceType":"volumeStoragePoolMap","topProcessName":"volume.purge","prettyProcess":"volume.purge->volumestoragepoolmap.remove","topResourceId":"4","processUuid":"34903daf-087f-455a-8cbf-0613ef0d85ac"}}
2015-01-15 14:31:18,238 INFO agent [139879102762032] [utils.py:351] worker45 : Starting request a3a1d249-4f2e-4220-b272-ba6f26e44092 for storage.volume.remove
2015-01-15 14:31:19,797 INFO agent [139879102762032] [utils.py:351] Response: {"name": "reply.3886858951383430044", "resourceType": "volumeStoragePoolMap", "resourceId": "4", "previousNames": ["storage.volume.remove"], "time": 1421332279000, "previousIds": ["a3a1d249-4f2e-4220-b272-ba6f26e44092"], "data": {"volume": {"format": "docker"}}, "id": "10d0b30c-a0dd-4d5a-829d-b0e21b4c3e0a"} [0.00483918190002] seconds
2015-01-15 14:31:19,797 INFO agent [139879102762032] [utils.py:351] worker45 : Done request a3a1d249-4f2e-4220-b272-ba6f26e44092 for storage.volume.remove [1.55951094627] seconds
2015-01-15 14:34:32,874 INFO agent [139879105408848] [utils.py:351] Request: {"id":"049201a9-9883-49bc-8dea-40b535d61429","name":"compute.instance.deactivate","replyTo":"reply.4119072804772911135","resourceId":"2","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":2,"instanceId":2,"hostId":1,"created":1421332174973,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"9129f41d-6053-4101-9ca7-40bd20b11710","name":null,"state":"deactivating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[2],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningProgress":90}},"token":"jTsh3Wi51gLhj2npel92uwATYo5MHcQFsvdWlBu6Khhl7r4VabrXWBc3doJMKAw5Q30LmLCNg0Q3flgGT9ZIg","id":2,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421332174789,"kind":"container","removeTime":null,"removed":null,"uuid":"24bc0c8e-3886-48d4-8008-9f6df502e6a0","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"stopping","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":3,"deviceNumber":0,"imageId":1,"instanceId":2,"offeringId":null,"zoneId":1,"accountId":1,"created":1421332174860,"kind":"volume","removeTime":null,"removed":null,"uuid":"388dcd2e-4288-4699-b4f0-7611587d0876","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[2],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningProgress":90}},"token":"jTsh3Wi51gLhj2npel92uwATYo5MHcQFsvdWlBu6Khhl7r4VabrXWBc3doJMKAw5Q30LmLCNg0Q3flgGT9ZIg","id":2,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421332174789,"kind":"container","removeTime":null,"removed":null,"uuid":"24bc0c8e-3886-48d4-8008-9f6df502e6a0","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"stopping","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":2,"deviceNumber":0,"instanceId":2,"macAddress":"02:77:36:2e:b8:d3","networkId":2,"subnetId":1,"vnetId":1,"accountId":1,"created":1421332174877,"kind":"nic","removeTime":null,"removed":null,"uuid":"085070f8-b5b6-4cd1-a4ca-c28178d89a82","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.25.105","data":{"fields":{}},"role":"primary","id":3,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":1,"created":1421332175240,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"8b8a2e07-93cc-47a1-aafd-74be0349b1aa","hostname":null,"name":"10.42.25.105","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999998,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332472846,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"44","logicName":"InstanceHostMapDeactivate","logicPath":"instance.stop->(InstanceStop)->instancehostmap.deactivate->(InstanceHostMapDeactivate)","processName":"instancehostmap.deactivate","prettyResource":"instance:2->instanceHostMap:2","topResourceType":"instance","resouceId":"2","resouceType":"instanceHostMap","topProcessName":"instance.stop","prettyProcess":"instance.stop->instancehostmap.deactivate","topResourceId":"2","processUuid":"6fa226fe-cacc-4acd-abf6-138eaa7d2cc7"}}
2015-01-15 14:34:32,901 INFO agent [139879105408848] [utils.py:351] worker25 : Starting request 049201a9-9883-49bc-8dea-40b535d61429 for compute.instance.deactivate
2015-01-15 14:34:32,941 INFO agent [139879105408848] [utils.py:351] Response: {"name": "reply.4119072804772911135", "resourceType": "instanceHostMap", "resourceId": "2", "previousNames": ["compute.instance.deactivate"], "time": 1421332472000, "previousIds": ["049201a9-9883-49bc-8dea-40b535d61429"], "data": {"instance": {"+data": {"dockerInspect": null, "dockerContainer": null, "+fields": {"dockerHostIp": "172.17.42.1", "dockerPorts": {}, "dockerIp": null}}}}, "id": "684844b0-8237-4d79-8a49-7fa499411056"} [0.0056619644165] seconds
2015-01-15 14:34:32,941 INFO agent [139879105408848] [utils.py:351] worker25 : Done request 049201a9-9883-49bc-8dea-40b535d61429 for compute.instance.deactivate [0.0401380062103] seconds
2015-01-15 14:34:33,104 INFO agent [139879105408208] [utils.py:351] Request: {"id":"6c75b8e0-ff20-4a25-95d7-8e19106dabdc","name":"config.update","replyTo":"reply.4119072804772911135","resourceId":"1","resourceType":"agent","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"configUrl":"http://172.17.0.12:8080/v1","items":[{"name":"host-iptables","requestedVersion":1,"apply":true,"increment":true,"checkInSyncOnly":false},{"name":"host-routes","requestedVersion":1,"apply":true,"increment":true,"checkInSyncOnly":false}]},"time":1421332473099,"timeoutMillis":2000,"transitioningProgress":null,"context":{"processId":"44","logicPath":"instance.stop->(InstanceStop)->instancehostmap.deactivate->(AgentInstanceHostServicesApplyItems)","logicName":"AgentInstanceHostServicesApplyItems","prettyResource":"instance:2->instanceHostMap:2","processName":"instancehostmap.deactivate","topResourceType":"instance","resouceId":"2","resouceType":"instanceHostMap","prettyProcess":"instance.stop->instancehostmap.deactivate","topProcessName":"instance.stop","topResourceId":"2","processUuid":"6fa226fe-cacc-4acd-abf6-138eaa7d2cc7"}}
2015-01-15 14:34:33,104 INFO agent [139879105408208] [utils.py:351] worker21 : Starting request 6c75b8e0-ff20-4a25-95d7-8e19106dabdc for config.update
2015-01-15 14:34:33,162 INFO agent [139879102761392] [utils.py:351] Request: {"id":"ce653ddd-1c0b-42ce-afc6-1df8d985ebe4","name":"storage.volume.deactivate","replyTo":"reply.4234254971626346161","resourceId":"3","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":3,"created":1421332174980,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"ecdd497b-d128-4adf-b51c-67cfd30a97a6","storagePoolId":1,"volumeId":3,"name":null,"state":"deactivating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":3,"deviceNumber":0,"imageId":1,"instanceId":2,"offeringId":null,"zoneId":1,"accountId":1,"created":1421332174860,"kind":"volume","removeTime":null,"removed":null,"uuid":"388dcd2e-4288-4699-b4f0-7611587d0876","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"deactivating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[2],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningProgress":90,"dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":null},"dockerInspect":null,"dockerContainer":null},"token":"jTsh3Wi51gLhj2npel92uwATYo5MHcQFsvdWlBu6Khhl7r4VabrXWBc3doJMKAw5Q30LmLCNg0Q3flgGT9ZIg","id":2,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421332174789,"kind":"container","removeTime":null,"removed":null,"uuid":"24bc0c8e-3886-48d4-8008-9f6df502e6a0","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"","state":"stopping","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332473156,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"44","logicName":"VolumeStoragePoolMapDeactivate","logicPath":"instance.stop->(InstanceStop)->volume.deactivate->(VolumeDeactivate)->volumestoragepoolmap.deactivate->(VolumeStoragePoolMapDeactivate)","processName":"volumestoragepoolmap.deactivate","prettyResource":"instance:2->volumeStoragePoolMap:3","topResourceType":"instance","resouceId":"3","resouceType":"volumeStoragePoolMap","topProcessName":"instance.stop","prettyProcess":"instance.stop->volumestoragepoolmap.deactivate","topResourceId":"2","processUuid":"6fa226fe-cacc-4acd-abf6-138eaa7d2cc7"}}
2015-01-15 14:34:33,163 INFO agent [139879102761392] [utils.py:351] worker41 : Starting request ce653ddd-1c0b-42ce-afc6-1df8d985ebe4 for storage.volume.deactivate
2015-01-15 14:34:33,167 INFO agent [139879102761392] [utils.py:351] Response: {"name": "reply.4234254971626346161", "resourceType": "volumeStoragePoolMap", "resourceId": "3", "previousNames": ["storage.volume.deactivate"], "time": 1421332473000, "previousIds": ["ce653ddd-1c0b-42ce-afc6-1df8d985ebe4"], "data": {"volume": {"format": "docker"}}, "id": "66c6c1ea-cebb-4a43-8591-6d662d5ad1cc"} [0.00409913063049] seconds
2015-01-15 14:34:33,167 INFO agent [139879102761392] [utils.py:351] worker41 : Done request ce653ddd-1c0b-42ce-afc6-1df8d985ebe4 for storage.volume.deactivate [0.00484108924866] seconds
2015-01-15 14:34:33,370 INFO agent [139879102760912] [utils.py:351] Request: {"id":"1ac291b1-88de-474c-847e-813bd3ba7c1d","name":"storage.volume.activate","replyTo":"reply.4234254971626346161","resourceId":"5","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":5,"created":1421332473178,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"07fd5e41-5255-4bad-83ee-bd1de40ffdd5","storagePoolId":1,"volumeId":5,"name":null,"state":"activating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":null,"uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"activating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create storage","transitioningProgress":10,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332473366,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"45","logicName":"VolumeStoragePoolMapActivate","logicPath":"instance.start->(InstanceStart)->volume.activate->(VolumeActivate)->volumestoragepoolmap.activate->(VolumeStoragePoolMapActivate)","processName":"volumestoragepoolmap.activate","prettyResource":"instance:4->volumeStoragePoolMap:5","topResourceType":"instance","resouceId":"5","resouceType":"volumeStoragePoolMap","topProcessName":"instance.start","prettyProcess":"instance.start->volumestoragepoolmap.activate","topResourceId":"4","processUuid":"8a411d98-a8c1-4507-8248-2a9f3f8b79c6"}}
2015-01-15 14:34:33,371 INFO agent [139879102760912] [utils.py:351] worker38 : Starting request 1ac291b1-88de-474c-847e-813bd3ba7c1d for storage.volume.activate
2015-01-15 14:34:33,377 INFO agent [139879102760912] [utils.py:351] Response: {"name": "reply.4234254971626346161", "resourceType": "volumeStoragePoolMap", "resourceId": "5", "previousNames": ["storage.volume.activate"], "time": 1421332473000, "previousIds": ["1ac291b1-88de-474c-847e-813bd3ba7c1d"], "data": {"volume": {"format": "docker"}}, "id": "56b1e493-96a9-436c-8300-c7d495d3b6ea"} [0.00591588020325] seconds
2015-01-15 14:34:33,377 INFO agent [139879102760912] [utils.py:351] worker38 : Done request 1ac291b1-88de-474c-847e-813bd3ba7c1d for storage.volume.activate [0.00657200813293] seconds
2015-01-15 14:34:33,420 INFO agent [139879102762512] [utils.py:351] Request: {"id":"177eb75d-5f2b-41e8-9822-ef01be6ad097","name":"compute.instance.activate","replyTo":"reply.4234254971626346161","resourceId":"4","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":4,"instanceId":4,"hostId":1,"created":1421332473169,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"10bacddc-42a2-4727-a9b2-2879b6d1b1da","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":4,"deviceNumber":0,"instanceId":4,"macAddress":"02:77:36:c6:54:f4","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332473063,"kind":"nic","removeTime":null,"removed":null,"uuid":"93a09c24-2735-4e52-ae11-44cb3f62dbdc","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.137.59","data":{"fields":{}},"role":"primary","id":5,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332473282,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"4170b975-c744-4cd4-af93-f7d9e7f918c4","hostname":null,"name":"10.42.137.59","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332473409,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"45","logicName":"InstanceHostMapActivate","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","processName":"instancehostmap.activate","prettyResource":"instance:4->instanceHostMap:4","topResourceType":"instance","resouceId":"4","resouceType":"instanceHostMap","topProcessName":"instance.start","prettyProcess":"instance.start->instancehostmap.activate","topResourceId":"4","processUuid":"8a411d98-a8c1-4507-8248-2a9f3f8b79c6"}}
2015-01-15 14:34:33,421 INFO agent [139879102762512] [utils.py:351] worker48 : Starting request 177eb75d-5f2b-41e8-9822-ef01be6ad097 for compute.instance.activate
2015-01-15 14:34:33,431 INFO docker [139879102762512] [compute.py:293] Creating docker container [4ed0d2b7-bc31-4269-88e3-6b8618951820] from config {'environment': {'CATTLE_CONFIG_URL': 'http://ipfacecobld28:8080/v1', u'CATTLE_AGENT_INSTANCE_AUTH': u'Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=='}, 'detach': True, 'name': u'4ed0d2b7-bc31-4269-88e3-6b8618951820', 'hostname': None}
2015-01-15 14:34:34,427 INFO agent [139879105408208] [utils.py:351] Response: {"name": "reply.4119072804772911135", "resourceType": "agent", "resourceId": "1", "previousNames": ["config.update"], "time": 1421332474000, "previousIds": ["6c75b8e0-ff20-4a25-95d7-8e19106dabdc"], "data": {"output": "INFO: Updating host-iptables\nINFO: Downloading http://ipfacecobld28:8080/v1//configcontent//host-iptables current=host-iptables-1-ab5c2eef05a68090f16b089f6243def6325a0ca4b33af7aa68aad2bee4b01290\nINFO: Already up to date\nINFO: Sending host-iptables applied 1-ab5c2eef05a68090f16b089f6243def6325a0ca4b33af7aa68aad2bee4b01290\nINFO: Updating host-routes\nINFO: Downloading http://ipfacecobld28:8080/v1//configcontent//host-routes current=host-routes-1-4ce17325c4239443229f7b8aaf22f8d29ee8f9142aed561f99b81dfd028999bd\nINFO: Already up to date\nINFO: Sending host-routes applied 1-4ce17325c4239443229f7b8aaf22f8d29ee8f9142aed561f99b81dfd028999bd\n", "exitCode": 0}, "id": "8a5b8124-75f6-436a-af9d-081570240fc3"} [0.00355219841003] seconds
2015-01-15 14:34:34,427 INFO agent [139879105408208] [utils.py:351] worker21 : Done request 6c75b8e0-ff20-4a25-95d7-8e19106dabdc for config.update [1.32327890396] seconds
2015-01-15 14:34:34,490 INFO docker [139879102762512] [compute.py:311] Starting docker container [4ed0d2b7-bc31-4269-88e3-6b8618951820] docker id [5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:34:35,225 ERROR cattle [139879102762512] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21200', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
52: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
53: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:12 brd ff:ff:ff:ff:ff:ff
52: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
53: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:12 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.18/16 scope global eth0
inet6 fe80::42:acff:fe11:12/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.137.59/16 to eth0
52: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
53: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:12 brd ff:ff:ff:ff:ff:ff
52: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
53: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:12 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.18/16 scope global eth0
inet 10.42.137.59/16 scope global eth0
inet6 fe80::42:acff:fe11:12/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:c6:54:f4
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21200', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:34:35,226 INFO agent [139879102762512] [utils.py:351] worker48 : Done request 177eb75d-5f2b-41e8-9822-ef01be6ad097 for compute.instance.activate [1.80557703972] seconds
2015-01-15 14:34:35,227 ERROR agent [139879102762512] [event.py:116] bfc7068f-bc35-4a98-b563-81cf2aa43953 : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21200', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:34:35,333 INFO agent [139879102762512] [utils.py:351] Response: {"name": "reply.4234254971626346161", "resourceType": "instanceHostMap", "resourceId": "4", "id": "006cb61f-80b5-4244-b208-68e6b35eec32", "transitioningInternalMessage": "bfc7068f-bc35-4a98-b563-81cf2aa43953 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21200', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332475000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["177eb75d-5f2b-41e8-9822-ef01be6ad097"]} [0.00449895858765] seconds
2015-01-15 14:34:47,893 INFO agent [139879102759472] [utils.py:351] Request: {"id":"5eadc116-7de4-48d7-9caa-9f67f4d80ba0","name":"compute.instance.activate","replyTo":"reply.2924108608311195223","resourceId":"4","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":4,"instanceId":4,"hostId":1,"created":1421332473169,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"10bacddc-42a2-4727-a9b2-2879b6d1b1da","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":4,"deviceNumber":0,"instanceId":4,"macAddress":"02:77:36:c6:54:f4","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332473063,"kind":"nic","removeTime":null,"removed":null,"uuid":"93a09c24-2735-4e52-ae11-44cb3f62dbdc","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.137.59","data":{"fields":{}},"role":"primary","id":5,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332473282,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"4170b975-c744-4cd4-af93-f7d9e7f918c4","hostname":null,"name":"10.42.137.59","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332487885,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"45","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:4->instanceHostMap:4","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"4","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"4","processUuid":"5223e152-5724-47ad-b3ec-d057a765273e"}}
2015-01-15 14:34:47,894 INFO agent [139879102759472] [utils.py:351] worker29 : Starting request 5eadc116-7de4-48d7-9caa-9f67f4d80ba0 for compute.instance.activate
2015-01-15 14:34:47,989 INFO docker [139879102759472] [compute.py:311] Starting docker container [4ed0d2b7-bc31-4269-88e3-6b8618951820] docker id [5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:34:48,979 ERROR cattle [139879102759472] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21725', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
56: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:13 brd ff:ff:ff:ff:ff:ff
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
56: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:13 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.19/16 scope global eth0
inet6 fe80::42:acff:fe11:13/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.137.59/16 to eth0
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
56: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:13 brd ff:ff:ff:ff:ff:ff
55: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
56: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:13 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.19/16 scope global eth0
inet 10.42.137.59/16 scope global eth0
inet6 fe80::42:acff:fe11:13/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:c6:54:f4
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21725', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:34:48,981 INFO agent [139879102759472] [utils.py:351] worker29 : Done request 5eadc116-7de4-48d7-9caa-9f67f4d80ba0 for compute.instance.activate [1.08768701553] seconds
2015-01-15 14:34:48,982 ERROR agent [139879102759472] [event.py:116] ddeb8a8b-a9dc-4a58-964f-a1be46888219 : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21725', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:34:48,991 INFO agent [139879102759472] [utils.py:351] Response: {"name": "reply.2924108608311195223", "resourceType": "instanceHostMap", "resourceId": "4", "id": "85b2f42e-dfab-4109-9fbf-336e35479d25", "transitioningInternalMessage": "ddeb8a8b-a9dc-4a58-964f-a1be46888219 : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '21725', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332488000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["5eadc116-7de4-48d7-9caa-9f67f4d80ba0"]} [0.00746393203735] seconds
2015-01-15 14:35:02,925 INFO agent [139879102761552] [utils.py:351] Request: {"id":"34a4468a-7523-4ebc-8ca4-cfb1b6c06914","name":"compute.instance.activate","replyTo":"reply.3883160542560393210","resourceId":"4","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":4,"instanceId":4,"hostId":1,"created":1421332473169,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"10bacddc-42a2-4727-a9b2-2879b6d1b1da","name":null,"state":"activating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningMessage":"create compute","transitioningProgress":90,"environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"starting","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":4,"deviceNumber":0,"instanceId":4,"macAddress":"02:77:36:c6:54:f4","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332473063,"kind":"nic","removeTime":null,"removed":null,"uuid":"93a09c24-2735-4e52-ae11-44cb3f62dbdc","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.137.59","data":{"fields":{}},"role":"primary","id":5,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332473282,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"4170b975-c744-4cd4-af93-f7d9e7f918c4","hostname":null,"name":"10.42.137.59","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332502917,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"45","logicPath":"instance.start->(InstanceStart)->instancehostmap.activate->(InstanceHostMapActivate)","logicName":"InstanceHostMapActivate","prettyResource":"instance:4->instanceHostMap:4","processName":"instancehostmap.activate","topResourceType":"instance","resouceId":"4","resouceType":"instanceHostMap","prettyProcess":"instance.start->instancehostmap.activate","topProcessName":"instance.start","topResourceId":"4","processUuid":"4571a0a0-e9e5-4a14-87ca-a7fe1969e4ce"}}
2015-01-15 14:35:02,926 INFO agent [139879102761552] [utils.py:351] worker42 : Starting request 34a4468a-7523-4ebc-8ca4-cfb1b6c06914 for compute.instance.activate
2015-01-15 14:35:02,952 INFO docker [139879102761552] [compute.py:311] Starting docker container [4ed0d2b7-bc31-4269-88e3-6b8618951820] docker id [5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b] {'publish_all_ports': False, 'port_bindings': {'4500/udp': ('0.0.0.0', 4500), '500/udp': ('0.0.0.0', 500)}, 'privileged': True}
2015-01-15 14:35:03,172 ERROR cattle [139879102761552] [utils.py:216] Failed to call (['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0'],) {'stderr': -2}, exit [2], output :
58: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
59: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
58: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
59: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.20/16 scope global eth0
inet6 fe80::42:acff:fe11:14/64 scope link tentative
valid_lft forever preferred_lft forever
Adding 10.42.137.59/16 to eth0
58: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
59: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 1000
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
58: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
59: eth0: <NO-CARRIER,BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN qlen 1000
link/ether 02:42:ac:11:00:14 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.20/16 scope global eth0
inet 10.42.137.59/16 scope global eth0
inet6 fe80::42:acff:fe11:14/64 scope link tentative
valid_lft forever preferred_lft forever
Setting eth0 to 02:77:36:c6:54:f4
RTNETLINK answers: Device or resource busy
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/utils.py", line 212, in get_command_output
return check_output(*args, **kw)
File "/var/lib/cattle/pyagent/cattle/utils.py", line 269, in check_output
raise e1
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:35:03,175 INFO agent [139879102761552] [utils.py:351] worker42 : Done request 34a4468a-7523-4ebc-8ca4-cfb1b6c06914 for compute.instance.activate [0.249300003052] seconds
2015-01-15 14:35:03,176 ERROR agent [139879102761552] [event.py:116] af9db6fd-1d86-407c-b78c-8eb90af975fe : Unknown error
Traceback (most recent call last):
File "/var/lib/cattle/pyagent/cattle/agent/event.py", line 99, in _worker_main
resp = agent.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/__init__.py", line 15, in execute
return self._router.route(req)
File "/var/lib/cattle/pyagent/cattle/plugins/core/event_router.py", line 13, in route
resp = handler.execute(req)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 33, in execute
return method(req=req, **req.data.__dict__)
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in instance_activate
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/agent/handler.py", line 71, in _do
action()
File "/var/lib/cattle/pyagent/cattle/compute/__init__.py", line 19, in <lambda>
action=lambda: self._do_instance_activate(instance, host, progress)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 314, in _do_instance_activate
self._call_listeners(False, instance, host, container['Id'])
File "/var/lib/cattle/pyagent/cattle/plugins/docker/compute.py", line 321, in _call_listeners
listener.after_start(*args)
File "/var/lib/cattle/pyagent/cattle/plugins/docker/network/setup.py", line 35, in after_start
device='eth{0}'.format(nic.deviceNumber))
File "/var/lib/cattle/pyagent/cattle/plugins/docker/util.py", line 40, in net_util
output = get_command_output(sudo(args))
File "/var/lib/cattle/pyagent/cattle/utils.py", line 217, in get_command_output
raise e
CalledProcessError: Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2
2015-01-15 14:35:03,183 INFO agent [139879102761552] [utils.py:351] Response: {"name": "reply.3883160542560393210", "resourceType": "instanceHostMap", "resourceId": "4", "id": "6ef57d34-f56c-4468-a7cb-3d52369eb91f", "transitioningInternalMessage": "af9db6fd-1d86-407c-b78c-8eb90af975fe : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2", "time": 1421332503000, "previousNames": ["compute.instance.activate"], "transitioning": "error", "data": {}, "previousIds": ["34a4468a-7523-4ebc-8ca4-cfb1b6c06914"]} [0.0051109790802] seconds
2015-01-15 14:35:03,305 INFO agent [139879102759632] [utils.py:351] Request: {"id":"174e4168-d02f-4cef-9ef4-b9b9c98b3c8c","name":"compute.instance.deactivate","replyTo":"reply.8664919765809893251","resourceId":"4","resourceType":"instanceHostMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"instanceHostMap":{"description":null,"data":{"fields":{}},"id":4,"instanceId":4,"hostId":1,"created":1421332473169,"kind":"instanceHostMap","removeTime":null,"removed":null,"uuid":"10bacddc-42a2-4727-a9b2-2879b6d1b1da","name":null,"state":"deactivating","type":"instanceHostMap","instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : af9db6fd-1d86-407c-b78c-8eb90af975fe : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","volumes":[{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"active","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : af9db6fd-1d86-407c-b78c-8eb90af975fe : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}}],"offering":null,"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"},"ports":[],"nics":[{"description":null,"data":{"fields":{}},"id":4,"deviceNumber":0,"instanceId":4,"macAddress":"02:77:36:c6:54:f4","networkId":2,"subnetId":1,"vnetId":1,"accountId":2,"created":1421332473063,"kind":"nic","removeTime":null,"removed":null,"uuid":"93a09c24-2735-4e52-ae11-44cb3f62dbdc","name":null,"state":"active","type":"nic","ipAddresses":[{"description":null,"address":"10.42.137.59","data":{"fields":{}},"role":"primary","id":5,"isPublic":false,"networkId":2,"ipPoolId":null,"subnetId":1,"accountId":2,"created":1421332473282,"kind":"ipAddress","removeTime":null,"removed":null,"uuid":"4170b975-c744-4cd4-af93-f7d9e7f918c4","hostname":null,"name":"10.42.137.59","state":"active","type":"ipAddress","subnet":{"description":null,"data":{"fields":{}},"id":1,"isPublic":true,"cidrSize":16,"networkAddress":"10.42.0.0","networkId":2,"gateway":"10.42.0.1","ipPoolId":null,"startAddress":"10.42.0.2","endAddress":"10.42.255.250","accountId":2,"created":1421331724255,"kind":"subnet","removeTime":null,"removed":null,"uuid":"docker0-subnet","name":"Subnet for managed docker0","state":"active","type":"subnet"}}],"network":{"domain":null,"description":null,"data":{"fields":{"hostVnetUri":"bridge://docker0","dynamicCreateVnet":true,"macPrefix":"02:77:36"},"libvirt":{"network":{"source":[{"bridge":"docker0"}],"type":"bridge"}}},"id":2,"isPublic":true,"accountId":2,"created":1421331724241,"kind":"hostOnlyNetwork","removeTime":null,"removed":null,"uuid":"managed-docker0","name":"Managed Network on docker0","state":"active","type":"network","networkServiceProviders":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724266,"kind":"agentInstanceProvider","removeTime":null,"removed":null,"uuid":"docker0-agent-instance-provider","name":"Agent instance provider for managed docker0","state":"active","type":"networkServiceProvider"}],"networkServices":[{"description":null,"data":{"fields":{}},"id":1,"networkId":2,"accountId":2,"created":1421331724276,"kind":"dnsService","removeTime":null,"removed":null,"uuid":"docker0-dns-service","networkServiceProviderId":1,"name":"DNS for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":2,"networkId":2,"accountId":2,"created":1421331724286,"kind":"dhcpService","removeTime":null,"removed":null,"uuid":"docker0-dhcp-service","networkServiceProviderId":1,"name":"DHCP for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":3,"networkId":2,"accountId":2,"created":1421331724289,"kind":"linkService","removeTime":null,"removed":null,"uuid":"docker0-link-service","networkServiceProviderId":1,"name":"Instance links for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":4,"networkId":2,"accountId":2,"created":1421331724293,"kind":"ipsecTunnelService","removeTime":null,"removed":null,"uuid":"docker0-ipsec-tunnel-service","networkServiceProviderId":1,"name":"IPsec tunnels for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":5,"networkId":2,"accountId":2,"created":1421331724296,"kind":"portService","removeTime":null,"removed":null,"uuid":"docker0-port-service","networkServiceProviderId":1,"name":"Ports service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":6,"networkId":2,"accountId":2,"created":1421331724300,"kind":"hostNatGatewayService","removeTime":null,"removed":null,"uuid":"docker0-host-nat-gateway-service","networkServiceProviderId":1,"name":"Host nat gateway service for managed docker0","state":"active","type":"networkService"},{"description":null,"data":{"fields":{}},"id":7,"networkId":2,"accountId":2,"created":1421331724304,"kind":"metadataService","removeTime":null,"removed":null,"uuid":"docker0-metadata-service","networkServiceProviderId":1,"name":"Meta data service for managed docker0","state":"active","type":"networkService"}]}}],"instanceLinks":[],"dataVolumesFromContainers":[]},"host":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"host","physicalHostUuid":"1cf98747-3289-48ac-b42e-ba785c5c5771"}},"uri":null,"id":1,"isPublic":false,"zoneId":1,"agentId":1,"physicalHostId":1,"accountId":4,"created":1421331749740,"kind":"docker","removeTime":null,"removed":null,"uuid":"f39b741c-24ee-486c-85e5-b2e29fa665aa","computeFree":999997,"computeTotal":1000000,"name":"ipfacecobld28","state":"active","type":"host"}}},"time":1421332503297,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"47","logicName":"InstanceHostMapDeactivate","logicPath":"instance.remove->(InstanceRemove)->instancehostmap.deactivate->(InstanceHostMapDeactivate)","processName":"instancehostmap.deactivate","prettyResource":"instance:4->instanceHostMap:4","topResourceType":"instance","resouceId":"4","resouceType":"instanceHostMap","topProcessName":"instance.remove","prettyProcess":"instance.remove->instancehostmap.deactivate","topResourceId":"4","processUuid":"d05f1cb3-71f8-4f35-9fbf-b494f6425fdd"}}
2015-01-15 14:35:03,306 INFO agent [139879102759632] [utils.py:351] worker30 : Starting request 174e4168-d02f-4cef-9ef4-b9b9c98b3c8c for compute.instance.deactivate
2015-01-15 14:35:04,485 INFO agent [139879102759632] [utils.py:351] Response: {"name": "reply.8664919765809893251", "resourceType": "instanceHostMap", "resourceId": "4", "previousNames": ["compute.instance.deactivate"], "time": 1421332504000, "previousIds": ["174e4168-d02f-4cef-9ef4-b9b9c98b3c8c"], "data": {"instance": {"+data": {"dockerInspect": {"AppArmorProfile": "", "Id": "5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b", "HostsPath": "/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hosts", "Created": "2015-01-15T14:34:33.861762766Z", "Image": "545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b", "Args": ["init"], "Driver": "devicemapper", "HostConfig": {"CapDrop": null, "ContainerIDFile": "", "NetworkMode": "", "Links": null, "LxcConf": null, "PortBindings": {"4500/udp": [{"HostIp": "0.0.0.0", "HostPort": "4500"}], "500/udp": [{"HostIp": "0.0.0.0", "HostPort": "500"}]}, "SecurityOpt": null, "CapAdd": null, "Binds": null, "RestartPolicy": {"MaximumRetryCount": 0, "Name": ""}, "Devices": null, "Dns": null, "ExtraHosts": null, "DnsSearch": null, "Privileged": true, "VolumesFrom": null, "PublishAllPorts": false}, "VolumesRW": {}, "State": {"Pid": 0, "Paused": false, "Running": false, "FinishedAt": "2015-01-15T14:35:04.405203054Z", "Restarting": false, "StartedAt": "2015-01-15T14:35:03.126189066Z", "ExitCode": -1}, "ExecDriver": "native-0.2", "ResolvConfPath": "/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/resolv.conf", "Volumes": {}, "Path": "/etc/init.d/agent-instance-startup", "HostnamePath": "/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hostname", "ProcessLabel": "", "Config": {"Volumes": {}, "Hostname": "5d432677c51e", "Entrypoint": null, "Env": ["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1", "CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==", "HOME=/", "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"], "Memory": 0, "OnBuild": null, "OpenStdin": false, "Cpuset": "", "User": "", "AttachStderr": false, "AttachStdout": false, "NetworkDisabled": false, "WorkingDir": "", "Cmd": ["/etc/init.d/agent-instance-startup", "init"], "StdinOnce": false, "PortSpecs": null, "AttachStdin": false, "MemorySwap": 0, "Tty": false, "CpuShares": 0, "Domainname": "", "Image": "rancher/agent-instance:v0.1.0", "ExposedPorts": {}}, "MountLabel": "", "NetworkSettings": {"MacAddress": "", "Bridge": "", "PortMapping": null, "IPPrefixLen": 0, "IPAddress": "", "Gateway": "", "Ports": null}, "Name": "/4ed0d2b7-bc31-4269-88e3-6b8618951820"}, "dockerContainer": {"Status": "Exited (-1) Less than a second ago", "Created": 1421332473, "Image": "rancher/agent-instance:v0.1.0", "Id": "5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b", "Command": "/etc/init.d/agent-instance-startup init", "Names": ["/4ed0d2b7-bc31-4269-88e3-6b8618951820"], "Ports": []}, "+fields": {"dockerHostIp": "172.17.42.1", "dockerPorts": {}, "dockerIp": ""}}}}, "id": "4b1aadf6-a34f-40fa-bcc3-e02aef2d534a"} [0.00572109222412] seconds
2015-01-15 14:35:04,486 INFO agent [139879102759632] [utils.py:351] worker30 : Done request 174e4168-d02f-4cef-9ef4-b9b9c98b3c8c for compute.instance.deactivate [1.18003082275] seconds
2015-01-15 14:35:04,531 INFO agent [139879105407248] [utils.py:351] Request: {"id":"22d1e475-5fb3-478d-b77b-24cba758fa18","name":"storage.volume.deactivate","replyTo":"reply.2739405191389171935","resourceId":"5","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":5,"created":1421332473178,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"07fd5e41-5255-4bad-83ee-bd1de40ffdd5","storagePoolId":1,"volumeId":5,"name":null,"state":"deactivating","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":null,"removed":null,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"active","name":null,"state":"deactivating","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"transitioningMessage":" : af9db6fd-1d86-407c-b78c-8eb90af975fe : Command '['/var/lib/cattle/pyagent/cattle/plugins/docker/net-util.sh', '-p', '22450', '-i', '10.42.137.59/16', '-m', u'02:77:36:c6:54:f4', '-d', 'eth0']' returned non-zero exit status 2","transitioning":"error","dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":"","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"dockerInspect":{"AppArmorProfile":"","Id":"5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b","HostsPath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hosts","Created":"2015-01-15T14:34:33.861762766Z","Image":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","Args":["init"],"Driver":"devicemapper","HostConfig":{"CapDrop":null,"ContainerIDFile":"","NetworkMode":"","Links":null,"LxcConf":null,"PortBindings":{"4500/udp":[{"HostIp":"0.0.0.0","HostPort":"4500"}],"500/udp":[{"HostIp":"0.0.0.0","HostPort":"500"}]},"SecurityOpt":null,"CapAdd":null,"Binds":null,"RestartPolicy":{"MaximumRetryCount":0,"Name":""},"Devices":null,"Dns":null,"ExtraHosts":null,"DnsSearch":null,"Privileged":true,"VolumesFrom":null,"PublishAllPorts":false},"VolumesRW":{},"State":{"Pid":0,"Paused":false,"Running":false,"FinishedAt":"2015-01-15T14:35:04.405203054Z","Restarting":false,"StartedAt":"2015-01-15T14:35:03.126189066Z","ExitCode":-1},"ExecDriver":"native-0.2","ResolvConfPath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/resolv.conf","Volumes":{},"Path":"/etc/init.d/agent-instance-startup","HostnamePath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hostname","ProcessLabel":"","Config":{"Volumes":{},"Hostname":"5d432677c51e","Entrypoint":null,"Env":["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1","CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==","HOME=/","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Memory":0,"OnBuild":null,"OpenStdin":false,"Cpuset":"","User":"","AttachStderr":false,"AttachStdout":false,"NetworkDisabled":false,"WorkingDir":"","Cmd":["/etc/init.d/agent-instance-startup","init"],"StdinOnce":false,"PortSpecs":null,"AttachStdin":false,"MemorySwap":0,"Tty":false,"CpuShares":0,"Domainname":"","Image":"rancher/agent-instance:v0.1.0","ExposedPorts":{}},"MountLabel":"","NetworkSettings":{"MacAddress":"","Bridge":"","PortMapping":null,"IPPrefixLen":0,"IPAddress":"","Gateway":"","Ports":null},"Name":"/4ed0d2b7-bc31-4269-88e3-6b8618951820"},"dockerContainer":{"Status":"Exited (-1) Less than a second ago","Created":1421332473,"Image":"rancher/agent-instance:v0.1.0","Id":"5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b","Command":"/etc/init.d/agent-instance-startup init","Names":["/4ed0d2b7-bc31-4269-88e3-6b8618951820"],"Ports":[]},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":null,"removed":null,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"active","hostname":null,"name":"Network Agent","state":"removing","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332504527,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"47","logicName":"VolumeStoragePoolMapDeactivate","logicPath":"instance.remove->(InstanceRemove)->volume.deactivate->(VolumeDeactivate)->volumestoragepoolmap.deactivate->(VolumeStoragePoolMapDeactivate)","processName":"volumestoragepoolmap.deactivate","prettyResource":"instance:4->volumeStoragePoolMap:5","topResourceType":"instance","resouceId":"5","resouceType":"volumeStoragePoolMap","topProcessName":"instance.remove","prettyProcess":"instance.remove->volumestoragepoolmap.deactivate","topResourceId":"4","processUuid":"d05f1cb3-71f8-4f35-9fbf-b494f6425fdd"}}
2015-01-15 14:35:04,531 INFO agent [139879105407248] [utils.py:351] worker15 : Starting request 22d1e475-5fb3-478d-b77b-24cba758fa18 for storage.volume.deactivate
2015-01-15 14:35:04,535 INFO agent [139879105407248] [utils.py:351] Response: {"name": "reply.2739405191389171935", "resourceType": "volumeStoragePoolMap", "resourceId": "5", "previousNames": ["storage.volume.deactivate"], "time": 1421332504000, "previousIds": ["22d1e475-5fb3-478d-b77b-24cba758fa18"], "data": {"volume": {"format": "docker"}}, "id": "8f01aefc-4c7f-4c44-b89f-6f5df5c88277"} [0.00374794006348] seconds
2015-01-15 14:35:04,535 INFO agent [139879105407248] [utils.py:351] worker15 : Done request 22d1e475-5fb3-478d-b77b-24cba758fa18 for storage.volume.deactivate [0.00433301925659] seconds
2015-01-15 14:36:18,591 INFO agent [139879105407728] [utils.py:351] Request: {"id":"d765ea61-9580-4602-9eda-a06eb1add40d","name":"storage.volume.remove","replyTo":"reply.2402963323535656915","resourceId":"5","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":5,"created":1421332473178,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"07fd5e41-5255-4bad-83ee-bd1de40ffdd5","storagePoolId":1,"volumeId":5,"name":null,"state":"removing","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":5,"deviceNumber":0,"imageId":2,"instanceId":4,"offeringId":null,"zoneId":1,"accountId":2,"created":1421332472989,"kind":"volume","removeTime":1421332564554,"removed":1421332504554,"uuid":"58df7f3e-01e1-4540-9992-2af5b215f756","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"deactivating","name":null,"state":"purging","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"vnetIds":[1],"privileged":true,"transitioningProgress":90,"dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":"","environment":{"CATTLE_AGENT_INSTANCE_AUTH":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="}},"ipsec":{"1":{"isakmp":500,"nat":4500}},"dockerInspect":{"AppArmorProfile":"","Id":"5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b","HostsPath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hosts","Created":"2015-01-15T14:34:33.861762766Z","Image":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","Args":["init"],"Driver":"devicemapper","HostConfig":{"CapDrop":null,"ContainerIDFile":"","NetworkMode":"","Links":null,"LxcConf":null,"PortBindings":{"4500/udp":[{"HostIp":"0.0.0.0","HostPort":"4500"}],"500/udp":[{"HostIp":"0.0.0.0","HostPort":"500"}]},"SecurityOpt":null,"CapAdd":null,"Binds":null,"RestartPolicy":{"MaximumRetryCount":0,"Name":""},"Devices":null,"Dns":null,"ExtraHosts":null,"DnsSearch":null,"Privileged":true,"VolumesFrom":null,"PublishAllPorts":false},"VolumesRW":{},"State":{"Pid":0,"Paused":false,"Running":false,"FinishedAt":"2015-01-15T14:35:04.405203054Z","Restarting":false,"StartedAt":"2015-01-15T14:35:03.126189066Z","ExitCode":-1},"ExecDriver":"native-0.2","ResolvConfPath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/resolv.conf","Volumes":{},"Path":"/etc/init.d/agent-instance-startup","HostnamePath":"/var/lib/docker/containers/5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b/hostname","ProcessLabel":"","Config":{"Volumes":{},"Hostname":"5d432677c51e","Entrypoint":null,"Env":["CATTLE_CONFIG_URL=http://ipfacecobld28:8080/v1","CATTLE_AGENT_INSTANCE_AUTH=Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg==","HOME=/","PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Memory":0,"OnBuild":null,"OpenStdin":false,"Cpuset":"","User":"","AttachStderr":false,"AttachStdout":false,"NetworkDisabled":false,"WorkingDir":"","Cmd":["/etc/init.d/agent-instance-startup","init"],"StdinOnce":false,"PortSpecs":null,"AttachStdin":false,"MemorySwap":0,"Tty":false,"CpuShares":0,"Domainname":"","Image":"rancher/agent-instance:v0.1.0","ExposedPorts":{}},"MountLabel":"","NetworkSettings":{"MacAddress":"","Bridge":"","PortMapping":null,"IPPrefixLen":0,"IPAddress":"","Gateway":"","Ports":null},"Name":"/4ed0d2b7-bc31-4269-88e3-6b8618951820"},"dockerContainer":{"Status":"Exited (-1) Less than a second ago","Created":1421332473,"Image":"rancher/agent-instance:v0.1.0","Id":"5d432677c51e106da6b3fb372b1f5d0ed77b4426d729fbc6c71370527fe65f3b","Command":"/etc/init.d/agent-instance-startup init","Names":["/4ed0d2b7-bc31-4269-88e3-6b8618951820"],"Ports":[]},"agentInstanceAuth":"Basic NTU0RDI3OTZFOTU2QkQ5NjYwQTQ6cXZwZEp4RExYa05rM2d1S0ZQblZzYmdBYTRUZ011bmhXVmJuNkNQQg=="},"token":"3lEadTKlYBpNbENNZPekF3qppNi53GKPOicmEitmglYEgFPya8LrLE9SDKOxCz7B9nURum01pK8tq8ANwcvQ","id":4,"imageId":2,"memoryMb":null,"userdata":null,"offeringId":null,"zoneId":1,"agentId":2,"instanceTriggeredStop":"restart","accountId":2,"created":1421332472963,"kind":"container","removeTime":1421332564558,"removed":1421332504558,"uuid":"4ed0d2b7-bc31-4269-88e3-6b8618951820","compute":null,"firstRunning":null,"allocationState":"inactive","hostname":null,"name":"Network Agent","state":"purged","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"545091ebf27543f7ddcbc8b061952ae2146f6fb110728e2fc65de6ec6a0b3a2b","repository":"agent-instance","namespace":"rancher","tag":"v0.1.0","qualifiedName":"rancher/agent-instance","fullName":"rancher/agent-instance:v0.1.0"},"fields":{}},"format":"docker","url":null,"id":2,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421332176904,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:rancher/agent-instance:v0.1.0","physicalSizeMb":null,"virtualSizeMb":null,"name":"rancher/agent-instance:v0.1.0","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332578582,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"52","logicName":"VolumeStoragePoolMapRemove","logicPath":"volume.purge->(VolumePurge)->volume.deallocate->(VolumeDeallocate)->volumestoragepoolmap.remove->(VolumeStoragePoolMapRemove)","processName":"volumestoragepoolmap.remove","prettyResource":"volume:5->volumeStoragePoolMap:5","topResourceType":"volume","resouceId":"5","resouceType":"volumeStoragePoolMap","topProcessName":"volume.purge","prettyProcess":"volume.purge->volumestoragepoolmap.remove","topResourceId":"5","processUuid":"84b35994-e367-42ba-af94-66c77f3968a4"}}
2015-01-15 14:36:18,592 INFO agent [139879105407728] [utils.py:351] worker18 : Starting request d765ea61-9580-4602-9eda-a06eb1add40d for storage.volume.remove
2015-01-15 14:36:20,862 INFO agent [139879105407728] [utils.py:351] Response: {"name": "reply.2402963323535656915", "resourceType": "volumeStoragePoolMap", "resourceId": "5", "previousNames": ["storage.volume.remove"], "time": 1421332580000, "previousIds": ["d765ea61-9580-4602-9eda-a06eb1add40d"], "data": {"volume": {"format": "docker"}}, "id": "e33876af-b20e-48d1-a33a-d82e49c58339"} [0.00818109512329] seconds
2015-01-15 14:36:20,863 INFO agent [139879105407728] [utils.py:351] worker18 : Done request d765ea61-9580-4602-9eda-a06eb1add40d for storage.volume.remove [2.27096605301] seconds
2015-01-15 14:37:18,674 INFO agent [139879105407728] [utils.py:351] Request: {"id":"90be6793-3746-489b-8ed0-ca92421576cf","name":"storage.volume.remove","replyTo":"reply.9003389044017051666","resourceId":"3","resourceType":"volumeStoragePoolMap","publisher":null,"transitioning":null,"transitioningMessage":null,"transitioningInternalMessage":null,"previousIds":null,"previousNames":null,"data":{"volumeStoragePoolMap":{"description":null,"data":{"fields":{}},"id":3,"created":1421332174980,"kind":"volumeStoragePoolMap","removeTime":null,"removed":null,"uuid":"ecdd497b-d128-4adf-b51c-67cfd30a97a6","storagePoolId":1,"volumeId":3,"name":null,"state":"removing","type":"volumeStoragePoolMap","volume":{"description":null,"data":{"fields":{}},"format":"docker","uri":null,"id":3,"deviceNumber":0,"imageId":1,"instanceId":2,"offeringId":null,"zoneId":1,"accountId":1,"created":1421332174860,"kind":"volume","removeTime":1421332580366,"removed":1421332520366,"uuid":"388dcd2e-4288-4699-b4f0-7611587d0876","attachedState":"active","physicalSizeMb":null,"virtualSizeMb":null,"allocationState":"deactivating","name":null,"state":"purging","type":"volume","offering":null,"instance":{"domain":null,"description":null,"data":{"fields":{"imageUuid":"docker:nginx","publishAllPorts":true,"startOnCreate":true,"dns":[],"lxcConf":{},"instanceLinks":{},"stdinOpen":false,"dnsSearch":[],"requestedHostId":1,"ports":[],"environment":{},"commandArgs":[],"networkIds":[2],"capAdd":[],"dataVolumesFrom":[],"tty":false,"memorySwap":null,"capDrop":[],"dataVolumes":[],"privileged":false,"transitioningProgress":90,"dockerHostIp":"172.17.42.1","dockerPorts":{},"dockerIp":null},"dockerInspect":null,"dockerContainer":null},"token":"jTsh3Wi51gLhj2npel92uwATYo5MHcQFsvdWlBu6Khhl7r4VabrXWBc3doJMKAw5Q30LmLCNg0Q3flgGT9ZIg","id":2,"imageId":1,"memoryMb":256,"userdata":null,"offeringId":null,"zoneId":1,"agentId":null,"instanceTriggeredStop":"stop","accountId":1,"created":1421332174789,"kind":"container","removeTime":1421332580370,"removed":1421332520370,"uuid":"24bc0c8e-3886-48d4-8008-9f6df502e6a0","compute":null,"firstRunning":null,"allocationState":"inactive","hostname":null,"name":"","state":"purged","type":"instance","dataVolumesFromContainers":[]},"storagePools":[{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}],"image":{"description":null,"data":{"dockerImage":{"id":"1822529acbbfb9983980b64750ad95f3245f5f504966a9234f3f99fff1acc064","repository":"nginx","namespace":null,"tag":"latest","qualifiedName":"nginx","fullName":"nginx:latest"},"fields":{}},"format":"docker","url":null,"id":1,"isPublic":true,"checksum":null,"instanceKind":"container","accountId":2,"created":1421331784002,"kind":"image","removeTime":null,"removed":null,"uuid":"docker:nginx","physicalSizeMb":null,"virtualSizeMb":null,"name":"nginx:latest","state":"active","type":"image"}},"storagePool":{"description":null,"data":{"fields":{"reportedUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306-pool","hostUuid":"c063f6ce-e164-43e0-ba01-9f64e30a6306","type":"storagePool"}},"id":1,"zoneId":1,"agentId":1,"accountId":4,"created":1421331749809,"kind":"docker","removeTime":null,"removed":null,"uuid":"4247ddd8-c564-492d-be32-e70e1102a2d5","external":false,"physicalTotalSizeMb":null,"virtualTotalSizeMb":null,"name":"ipfacecobld28 Storage Pool","state":"active","type":"storagePool"}}},"time":1421332638667,"timeoutMillis":15000,"transitioningProgress":null,"context":{"processId":"56","logicName":"VolumeStoragePoolMapRemove","logicPath":"volume.purge->(VolumePurge)->volume.deallocate->(VolumeDeallocate)->volumestoragepoolmap.remove->(VolumeStoragePoolMapRemove)","processName":"volumestoragepoolmap.remove","prettyResource":"volume:3->volumeStoragePoolMap:3","topResourceType":"volume","resouceId":"3","resouceType":"volumeStoragePoolMap","topProcessName":"volume.purge","prettyProcess":"volume.purge->volumestoragepoolmap.remove","topResourceId":"3","processUuid":"1e607598-d47c-4503-b482-3f04140fb45a"}}
2015-01-15 14:37:18,674 INFO agent [139879105407728] [utils.py:351] worker18 : Starting request 90be6793-3746-489b-8ed0-ca92421576cf for storage.volume.remove
2015-01-15 14:37:18,697 INFO agent [139879105407728] [utils.py:351] Response: {"name": "reply.9003389044017051666", "resourceType": "volumeStoragePoolMap", "resourceId": "3", "previousNames": ["storage.volume.remove"], "time": 1421332638000, "previousIds": ["90be6793-3746-489b-8ed0-ca92421576cf"], "data": {"volume": {"format": "docker"}}, "id": "824de82e-5e13-4f01-a088-1a53fc7e6965"} [0.0152018070221] seconds
2015-01-15 14:37:18,697 INFO agent [139879105407728] [utils.py:351] worker18 : Done request 90be6793-3746-489b-8ed0-ca92421576cf for storage.volume.remove [0.0233240127563] seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment