Recently, Everyday Hero has been doing a heap of work around the automation and provisioning of resources with AWS cloud services. This entails working frequently with the AWS API.
A useful filter we have been using in Ansible is taking output from a shell action and turning it into something we can consume via variables.
An example playbook is below:
- shell: |
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_KEY
lib/ec2.py --list --refresh-cache
register: output
- set_fact:
ec2_output: "{{ output.stdout|from_json }}"
- shell: "rm addresses.txt"
- shell: "echo '{{ item.value['ec2_private_ip_address'] }}' >> addresses.txt"
when: item.value["ec2_tag_Name"] is defined and item.value["ec2_tag_Name"] == "deis-{{ environment_unique_id }}"
with_dict: ec2_output._meta.hostvars
- shell: "cat addresses.txt"
register: output
- add_host:
hostname: "{{ output.stdout_lines.0 }}"
groupname: "deis_node"
- debug: var={{ groups['deis_node'].0 }}
This example uses an Ansible plugin script ec2.py to retrieve a list of instances
running in an AWS account. It then filters out the ones that we want via the
ec2_tag_Name
key and adds the private IP address to a file.
After running the following shell action, output
holds a heap of information
about what the commands did and the output from stdout and stderr (if there was
any.)
- shell: |
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY
export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_KEY
lib/ec2.py --list --refresh-cache
register: output
The Ansible output for the registered variable output looks something like this:
TASK: [debug var=output] ******************************************************
ok: [localhost] => {
"output": {
"changed": true,
"cmd": "export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY \nexport AWS_SECRET_ACCESS_KEY=$AWS_SECRET_KEY \nlib/ec2.py --list --refresh-cache",
"delta": "0:00:05.268682",
"end": "2014-10-05 11:43:05.235594",
"invocation": {
"module_args": "export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY \nexport AWS_SECRET_ACCESS_KEY=$AWS_SECRET_KEY \nlib/ec2.py --list --refresh-cache",
"module_name": "shell"
},
"rc": 0,
"start": "2013-10-05 11:42:59.966912",
"stderr": "",
"stdout": "<suppressed to keep it short>",
"stdout_lines": [
"{",
" \"_meta\": {",
" \"hostvars\": {",
" \"xx.xxx.xxx.xxx\": {",
" \"ec2__in_monitoring_element\": false, ",
" \"ec2_ami_launch_index\": \"0\", ",
" \"ec2_architecture\": \"x86_64\", ",
" \"ec2_client_token\": \"xxxxxxxx-xxxx-xxxx-xxxx-c55977fc0029_us-east-1a_1\", ",
" \"ec2_dns_name\": \"ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com\", ",
" \"ec2_ebs_optimized\": false, ",
" \"ec2_eventsSet\": \"\", ",
" \"ec2_group_name\": \"\", ",
" \"ec2_hypervisor\": \"xen\", ",
" \"ec2_id\": \"i-xxxxxx\", ",
" \"ec2_image_id\": \"ami-xxxxxx\", ",
" \"ec2_instance_profile\": \"\", ",
" \"ec2_instance_type\": \"m1.large\", ",
" \"ec2_ip_address\": \"xx.xxx.xxx.xxx\", ",
" \"ec2_item\": \"\", ",
" \"ec2_kernel\": \"aki-xxxxxxxx\", ",
" \"ec2_key_name\": \"ambari\", ",
" \"ec2_launch_time\": \"2013-10-01T04:53:00.000Z\", ",
" \"ec2_monitored\": true, ",
" \"ec2_monitoring\": \"\", ",
" \"ec2_monitoring_state\": \"enabled\", ",
" \"ec2_persistent\": false, ",
" \"ec2_placement\": \"us-east-1a\", ",
" \"ec2_platform\": \"\", ",
" \"ec2_previous_state\": \"\", ",
" \"ec2_previous_state_code\": 0, ",
" \"ec2_private_dns_name\": \"ip-xx-xx-x-xxx.ec2.internal\", ",
" \"ec2_private_ip_address\": \"xx.xx.x.xxx\", ",
" \"ec2_public_dns_name\": \"ec2-xx-xxx-xxx-xxx.compute-1.amazonaws.com\", ",
" \"ec2_ramdisk\": \"\", ",
" \"ec2_reason\": \"\", ",
" \"ec2_region\": \"us-east-1\", ",
" \"ec2_requester_id\": \"\", ",
" \"ec2_root_device_name\": \"/dev/sda1\", ",
" \"ec2_root_device_type\": \"ebs\", ",
" \"ec2_security_group_ids\": \"sg-xxxxxxxx\", ",
" \"ec2_security_group_names\": \"ambari\", ",
" \"ec2_sourceDestCheck\": \"true\", ",
" \"ec2_spot_instance_request_id\": \"\", ",
" \"ec2_state\": \"running\", ",
" \"ec2_state_code\": 16, ",
" \"ec2_state_reason\": \"\", ",
" \"ec2_subnet_id\": \"subnet-xxxxxxxx\", ",
" \"ec2_tag_Name\": \"hdpmaster1\", ",
" \"ec2_tag_aws_autoscaling_groupName\": \"ambari-LargeClusterGroup-148W3OVR9LSSE\", ",
" \"ec2_tag_aws_cloudformation_logical-id\": \"LargeClusterGroup\", ",
" \"ec2_tag_aws_cloudformation_stack-id\": \"arn:aws:cloudformation:us-east-1:167609138788:stack/ambari/xxxxxxxx-xxxx-xxxx-xxxx-50fa526be49c\", ",
" \"ec2_tag_aws_cloudformation_stack-name\": \"ambari\", ",
" \"ec2_tag_long_hostname\": \"hdpmaster1.domain.com\", ",
" \"ec2_virtualization_type\": \"paravirtual\", ",
" \"ec2_vpc_id\": \"vpc-xxxxxxxx\"",
" }, ",
"}"
]
}
}
As you can see stdout and stdout_lines aren't in a great format to do much with, here enters from_json:
- set_fact:
ec2_output: "{{ output.stdout|from_json }}"
This then allows us to do the following and ec2_output
is now in a standard
JSON format that Ansible can then use to iterate over like so:
- shell: "echo '{{ item.value['ec2_private_ip_address'] }}' >> addresses.txt"
when: item.value["ec2_tag_Name"] is defined and item.value["ec2_tag_Name"] == "deis-{{ environment_unique_id }}"
with_dict: ec2_output._meta.hostvars
This Ansible task creates a file that contains all of the private IP addresses
of ec2 instances that match the ec2_tag_Name
criteria.
We could then take the first of these IP addresses and use it later in our playbook:
- shell: "cat addresses.txt"
register: output
- debug: var={{ output.stdout_lines.0 }}
This becomes a very powerful pattern when you mix in RESTful APIs, calls to them and processing output from them.
thank you! even tho my problem was completely different, syntax that you used was what I needed!