Last active
October 24, 2022 19:22
-
-
Save gene1wood/a00d0b9d029f40e866df to your computer and use it in GitHub Desktop.
A CloudFormation template for CentOS 7 showing AMI mappings and cloud-init UserData parsing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| AWSTemplateFormatVersion: 2010-09-09 | |
| Description: > | |
| An example free tier (12 month) CentOS 7 EC2 instance with a security group | |
| allowing SSH, a simple IAM Role, and output conveyed from the launch back out | |
| to the CloudFormation stack outputs | |
| Parameters: | |
| SSHKeyName: | |
| Description: SSH Key Name | |
| Type: String | |
| Mappings: | |
| # This list of AMI IDs was produced with this code | |
| # https://gist.github.com/gene1wood/56e42097e0f0ac1aace14cbc41ee3e11#file-create_centos7_cloudformation_ami_mapping-py | |
| RegionMap: | |
| ap-northeast-1: | |
| CentOS7x8664EBSHVM: ami-045f38c93733dd48d | |
| ap-northeast-2: | |
| CentOS7x8664EBSHVM: ami-06cf2a72dadf92410 | |
| ap-south-1: | |
| CentOS7x8664EBSHVM: ami-02e60be79e78fef21 | |
| ap-southeast-1: | |
| CentOS7x8664EBSHVM: ami-0b4dd9d65556cac22 | |
| ap-southeast-2: | |
| CentOS7x8664EBSHVM: ami-08bd00d7713a39e7d | |
| ca-central-1: | |
| CentOS7x8664EBSHVM: ami-033e6106180a626d0 | |
| eu-central-1: | |
| CentOS7x8664EBSHVM: ami-04cf43aca3e6f3de3 | |
| eu-north-1: | |
| CentOS7x8664EBSHVM: ami-5ee66f20 | |
| eu-west-1: | |
| CentOS7x8664EBSHVM: ami-0ff760d16d9497662 | |
| eu-west-2: | |
| CentOS7x8664EBSHVM: ami-0eab3a90fc693af19 | |
| eu-west-3: | |
| CentOS7x8664EBSHVM: ami-0e1ab783dc9489f34 | |
| sa-east-1: | |
| CentOS7x8664EBSHVM: ami-0b8d86d4bf91850af | |
| us-east-1: | |
| CentOS7x8664EBSHVM: ami-02eac2c0129f6376b | |
| us-east-2: | |
| CentOS7x8664EBSHVM: ami-0f2b4fc905b0bd1f1 | |
| us-west-1: | |
| CentOS7x8664EBSHVM: ami-074e2d6769f445be5 | |
| us-west-2: | |
| CentOS7x8664EBSHVM: ami-01ed306a12b7d1c96 | |
| Resources: | |
| SecurityGroup: | |
| Type: AWS::EC2::SecurityGroup | |
| Properties: | |
| GroupDescription: Security Group | |
| SecurityGroupIngress: | |
| - FromPort: 22 | |
| ToPort: 22 | |
| IpProtocol: tcp | |
| CidrIp: '0.0.0.0/0' | |
| Instance: | |
| Type: AWS::EC2::Instance | |
| Metadata: | |
| UserDataComment1: The cloud-config script is delivered directly in user_data | |
| because the CentOS 7 base AMIs run cloud-init automatically, looking for a | |
| cloud-init config in user_data | |
| UserDataComment2: In this example the 'dmidecode' command is run and the output | |
| is sent back to the WaitConditionHandle which is then passed out through the | |
| CloudFormation outputs | |
| Properties: | |
| ImageId: !FindInMap [ RegionMap, !Ref 'AWS::Region', CentOS7x8664EBSHVM ] | |
| InstanceType: t2.micro # Instance size of the free tier for 12 months | |
| BlockDeviceMappings: | |
| - DeviceName: /dev/sda1 | |
| Ebs: | |
| VolumeType: gp2 | |
| DeleteOnTermination: false | |
| VolumeSize: 30 # Free tier provides 30GiB of gp2 or standard EBS | |
| KeyName: !Ref SSHKeyName | |
| SecurityGroups: | |
| - !Ref SecurityGroup | |
| IamInstanceProfile: !Ref InstanceProfile | |
| UserData: | |
| Fn::Base64: !Sub | | |
| #cloud-config | |
| packages: | |
| - awscli | |
| runcmd: | |
| - > | |
| for i in {1..3}; do /usr/bin/easy_install | |
| https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz | |
| 2>&1 >> /var/log/initial_user-data.log && break || sleep 10; done | |
| - aws --region us-west-2 ec2 describe-regions --output text >> /tmp/custom-output.txt | |
| - CFNSTATUS=$? | |
| - > | |
| /usr/bin/cfn-signal | |
| --exit-code $CFNSTATUS | |
| --data "$( < /tmp/custom-output.txt )" | |
| "${WaitConditionHandle}" 2>&1 >> /var/log/initial_user-data.log | |
| Role: | |
| Type: AWS::IAM::Role | |
| Properties: | |
| AssumeRolePolicyDocument: | |
| Version: 2012-10-17 | |
| Statement: | |
| - Effect: Allow | |
| Principal: | |
| Service: ec2.amazonaws.com | |
| Action: sts:AssumeRole | |
| Policies: | |
| - PolicyName: DescribeRegions | |
| PolicyDocument: | |
| Version: 2012-10-17 | |
| Statement: | |
| - Sid: AllowEC2DescribeRegions | |
| Effect: Allow | |
| Action: | |
| - ec2:DescribeRegions | |
| Resource: '*' | |
| InstanceProfile: | |
| Type: AWS::IAM::InstanceProfile | |
| Properties: | |
| Roles: | |
| - !Ref Role | |
| WaitConditionHandle: | |
| Type: AWS::CloudFormation::WaitConditionHandle | |
| WaitCondition: | |
| Type: AWS::CloudFormation::WaitCondition | |
| DependsOn: Instance | |
| Properties: | |
| Handle: !Ref WaitConditionHandle | |
| Timeout: '300' | |
| Outputs: | |
| CloudInitOutput: | |
| Description: The data returned to the WaitConditionHandle from Cloud Init | |
| Value: !GetAtt WaitCondition.Data | |
| EC2InstancePublicDNSName: | |
| Description: The public DNS name of the instance | |
| Value: !GetAtt Instance.PublicDnsName |
@josdirksen, thanks for catching that! I've fixed it above at 3812576afe34046def330757c99a632bd127ed54
Not sure if this was intentional, but there are two AIM missing in your mapping:
Asia Pacific (Mumbai) ami-95cda6fa
US East (Ohio) ami-6a2d760f
I tried this, as written, and it times out and rolls back. I could sure use some help understanding what is failing. The log files are below.
initial_user-data.log
Downloading https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
Processing aws-cfn-bootstrap-latest.tar.gz
Writing /tmp/easy_install-cHTPlA/aws-cfn-bootstrap-1.4/setup.cfg
Running aws-cfn-bootstrap-1.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-cHTPlA/aws-cfn-bootstrap-1.4/egg-dist-tmp-LR21FA
Adding aws-cfn-bootstrap 1.4 to easy-install.pth file
Installing cfn-init script to /usr/bin
Installing cfn-signal script to /usr/bin
Installing cfn-get-metadata script to /usr/bin
Installing cfn-hup script to /usr/bin
Installing cfn-elect-cmd-leader script to /usr/bin
Installing cfn-send-cmd-result script to /usr/bin
Installing cfn-send-cmd-event script to /usr/bin
Installed /usr/lib/python2.7/site-packages/aws_cfn_bootstrap-1.4-py2.7.egg
Processing dependencies for aws-cfn-bootstrap==1.4
Searching for pystache>=0.4.0
Reading https://pypi.python.org/simple/pystache/
Best match: pystache 0.5.4
Downloading https://pypi.python.org/packages/d6/fd/eb8c212053addd941cc90baac307c00ac246ac3fce7166b86434c6eae963/pystache-0.5.4.tar.gz#md5=485885e67a0f6411d5252e69b20a35ca
Processing pystache-0.5.4.tar.gz
Writing /tmp/easy_install-4XGsWd/pystache-0.5.4/setup.cfg
Running pystache-0.5.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-4XGsWd/pystache-0.5.4/egg-dist-tmp-dli6k7
Adding pystache 0.5.4 to easy-install.pth file
Installing pystache script to /usr/bin
Installing pystache-test script to /usr/bin
Installed /usr/lib/python2.7/site-packages/pystache-0.5.4-py2.7.egg
Searching for python-daemon>=1.5.2,<2.0
Reading https://pypi.python.org/simple/python-daemon/
Best match: python-daemon 1.6.1
Downloading https://pypi.python.org/packages/b8/e2/bd862c723451121f41cf53d87a9f2c69eeb97358fb4477b929878804b8a3/python-daemon-1.6.1.tar.gz#md5=cbad567c105878776b567535a8b7372a
Processing python-daemon-1.6.1.tar.gz
Writing /tmp/easy_install-dl799F/python-daemon-1.6.1/setup.cfg
Running python-daemon-1.6.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-dl799F/python-daemon-1.6.1/egg-dist-tmp-IZWRk7
Adding python-daemon 1.6.1 to easy-install.pth file
Installed /usr/lib/python2.7/site-packages/python_daemon-1.6.1-py2.7.egg
Searching for lockfile>=0.9
Reading https://pypi.python.org/simple/lockfile/
Best match: lockfile 0.12.2
Downloading https://pypi.python.org/packages/17/47/72cb04a58a35ec495f96984dddb48232b551aafb95bde614605b754fe6f7/lockfile-0.12.2.tar.gz#md5=a6a1a82957a23afdf44cfdd039b65ff9
Processing lockfile-0.12.2.tar.gz
Writing /tmp/easy_install-8mxYNh/lockfile-0.12.2/setup.cfg
Running lockfile-0.12.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-8mxYNh/lockfile-0.12.2/egg-dist-tmp-rfLQ7a
Searching for pbr>=1.8
Reading https://pypi.python.org/simple/pbr/
Best match: pbr 3.1.1
Downloading https://pypi.python.org/packages/d5/d6/f2bf137d71e4f213b575faa9eb426a8775732432edb67588a8ee836ecb80/pbr-3.1.1.tar.gz#md5=4e82c2e07af544c56a5b71c801525b00
Processing pbr-3.1.1.tar.gz
Writing /tmp/easy_install-8mxYNh/lockfile-0.12.2/temp/easy_install-JtYloG/pbr-3.1.1/setup.cfg
Running pbr-3.1.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-8mxYNh/lockfile-0.12.2/temp/easy_install-JtYloG/pbr-3.1.1/egg-dist-tmp-IVRvhU
Installed /tmp/easy_install-8mxYNh/lockfile-0.12.2/pbr-3.1.1-py2.7.egg
Adding lockfile 0.12.2 to easy-install.pth file
Installed /usr/lib/python2.7/site-packages/lockfile-0.12.2-py2.7.egg
Finished processing dependencies for aws-cfn-bootstrap==1.4
cloud-init.txt
Aug 22 00:22:57 localhost cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Tue, 22 Aug 2017 00:22:57 +0000. Up 18.69 seconds.
Aug 22 00:23:00 localhost cloud-init: Cloud-init v. 0.7.5 running 'init' at Tue, 22 Aug 2017 00:23:00 +0000. Up 21.32 seconds.
Aug 22 00:23:00 localhost cloud-init: ci-info: ++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++
Aug 22 00:23:00 localhost cloud-init: ci-info: +--------+------+--------------+---------------+-------------------+
Aug 22 00:23:01 localhost cloud-init: ci-info: | Device | Up | Address | Mask | Hw-Address |
Aug 22 00:23:01 localhost cloud-init: ci-info: +--------+------+--------------+---------------+-------------------+
Aug 22 00:23:01 localhost cloud-init: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
Aug 22 00:23:01 localhost cloud-init: ci-info: | eth0: | True | 172.31.44.94 | 255.255.240.0 | 0a:dd:94:e6:5b:a2 |
Aug 22 00:23:01 localhost cloud-init: ci-info: +--------+------+--------------+---------------+-------------------+
Aug 22 00:23:01 localhost cloud-init: ci-info: +++++++++++++++++++++++++++++++Route info++++++++++++++++++++++++++++++++
Aug 22 00:23:01 localhost cloud-init: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
Aug 22 00:23:01 localhost cloud-init: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
Aug 22 00:23:01 localhost cloud-init: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
Aug 22 00:23:01 localhost cloud-init: ci-info: | 0 | 0.0.0.0 | 172.31.32.1 | 0.0.0.0 | eth0 | UG |
Aug 22 00:23:01 localhost cloud-init: ci-info: | 1 | 172.31.32.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U |
Aug 22 00:23:01 localhost cloud-init: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
Aug 22 00:23:05 localhost cloud-init: Cloud-init v. 0.7.5 running 'modules:config' at Tue, 22 Aug 2017 00:23:05 +0000. Up 26.72 seconds.
Aug 22 00:23:06 localhost cloud-init: Cloud-init v. 0.7.5 running 'modules:final' at Tue, 22 Aug 2017 00:23:06 +0000. Up 27.40 seconds.
Aug 22 00:23:08 localhost cloud-init: zip_safe flag not set; analyzing archive contents...
Aug 22 00:23:08 localhost cloud-init: cfnbootstrap.packages.requests.certs: module references __file__
Aug 22 00:23:08 localhost cloud-init: cfnbootstrap.packages.requests.certs: module references __file__
Aug 22 00:23:09 localhost cloud-init: pystache: using: version '0.9.8' of <module 'setuptools' from '/usr/lib/python2.7/site-packages/setuptools/__init__.pyc'>
Aug 22 00:23:09 localhost cloud-init: zip_safe flag not set; analyzing archive contents...
Aug 22 00:23:09 localhost cloud-init: pystache.locator: module references __file__
Aug 22 00:23:09 localhost cloud-init: pystache.tests.common: module references __file__
Aug 22 00:23:12 localhost cloud-init: Usage: cfn-signal [options] [WaitConditionHandle URL]
Aug 22 00:23:12 localhost cloud-init: cfn-signal: error: option --exit-code: invalid integer value: '--data'
Aug 22 00:23:12 localhost cloud-init: 2017-08-22 00:23:12,378 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/runcmd [2]
Aug 22 00:23:12 localhost cloud-init: 2017-08-22 00:23:12,406 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
Aug 22 00:23:12 localhost cloud-init: 2017-08-22 00:23:12,407 - util.py[WARNING]: Running scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Aug 22 00:23:13 localhost cloud-init: Cloud-init v. 0.7.5 finished at Tue, 22 Aug 2017 00:23:13 +0000. Datasource DataSourceEc2. Up 34.45 seconds
cloud-init-output.txt
Cloud-init v. 0.7.5 running 'init-local' at Tue, 22 Aug 2017 00:22:57 +0000. Up 18.69 seconds.
Cloud-init v. 0.7.5 running 'init' at Tue, 22 Aug 2017 00:23:00 +0000. Up 21.32 seconds.
ci-info: ++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++
ci-info: +--------+------+--------------+---------------+-------------------+
ci-info: | Device | Up | Address | Mask | Hw-Address |
ci-info: +--------+------+--------------+---------------+-------------------+
ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
ci-info: | eth0: | True | 172.31.44.94 | 255.255.240.0 | 0a:dd:94:e6:5b:a2 |
ci-info: +--------+------+--------------+---------------+-------------------+
ci-info: +++++++++++++++++++++++++++++++Route info++++++++++++++++++++++++++++++++
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
ci-info: | 0 | 0.0.0.0 | 172.31.32.1 | 0.0.0.0 | eth0 | UG |
ci-info: | 1 | 172.31.32.0 | 0.0.0.0 | 255.255.240.0 | eth0 | U |
ci-info: +-------+-------------+-------------+---------------+-----------+-------+
Cloud-init v. 0.7.5 running 'modules:config' at Tue, 22 Aug 2017 00:23:05 +0000. Up 26.72 seconds.
Cloud-init v. 0.7.5 running 'modules:final' at Tue, 22 Aug 2017 00:23:06 +0000. Up 27.40 seconds.
zip_safe flag not set; analyzing archive contents...
cfnbootstrap.packages.requests.certs: module references __file__
cfnbootstrap.packages.requests.certs: module references __file__
pystache: using: version '0.9.8' of <module 'setuptools' from '/usr/lib/python2.7/site-packages/setuptools/__init__.pyc'>
zip_safe flag not set; analyzing archive contents...
pystache.locator: module references __file__
pystache.tests.common: module references __file__
Usage: cfn-signal [options] [WaitConditionHandle URL]
cfn-signal: error: option --exit-code: invalid integer value: '--data'
2017-08-22 00:23:12,378 - util.py[WARNING]: Failed running /var/lib/cloud/instance/scripts/runcmd [2]
2017-08-22 00:23:12,406 - cc_scripts_user.py[WARNING]: Failed to run module scripts-user (scripts in /var/lib/cloud/instance/scripts)
2017-08-22 00:23:12,407 - util.py[WARNING]: Running scripts-user (<module 'cloudinit.config.cc_scripts_user' from '/usr/lib/python2.7/site-packages/cloudinit/config/cc_scripts_user.pyc'>) failed
Cloud-init v. 0.7.5 finished at Tue, 22 Aug 2017 00:23:13 +0000. Datasource DataSourceEc2. Up 34.45 seconds
I found what's wrong with the above. The clue is in the line cfn-signal: error: option --exit-code: invalid integer value: '--data'
In the template, we have this:
"UserData":{
"Fn::Base64":{
"Fn::Join":[
"",
[
"#cloud-config\n",
"\n",
"runcmd:\n",
" - for i in {1..3}; do /usr/bin/easy_install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz 2>&1 >> /var/log/initial_user-data.log && break || sleep 10; done\n",
" - dmidecode 2>&1 >> /tmp/custom-output.txt",
" - CFNSTATUS=$?\n",
" - /usr/bin/cfn-signal ",
" --exit-code $CFNSTATUS ",
" --data \"$( base64 -w 0 /tmp/custom-output.txt )\" ",
"'",
{
"Ref":"WaitConditionHandle"
},
"'",
" 2>&1 >> /var/log/initial_user-data.log\n"
]
]
}
}
It looks like $CFNSTATUS is not being set to 0 (or an error), but rather is empty. I proved this by adding a line to echo $CFNSTATUS into a file in /tmp, and also to provide a 0 to the --exit-code argument in the call to /usr/bin/cfn-signal.
Once I made those changes, the server built.
Not sure why this would be so.
@edgreenberg Good catch. The reason for that bug was that the line that runs dmidecode doesn't have a trailing \n. So the line that looks like this
" - dmidecode 2>&1 >> /tmp/custom-output.txt",
should have looked like this
" - dmidecode 2>&1 >> /tmp/custom-output.txt\n",
I'm updating this template to yaml which will solve this problem.
@locomotif The updated template contains the regions you'd mentioned. The AMI mapping is generated with code that you can find here (so that it won't miss any regions) : https://gist.github.com/gene1wood/56e42097e0f0ac1aace14cbc41ee3e11#file-create_centos7_cloudformation_ami_mapping-py
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There is a bug in line 70:
"CentOS7x8664withUpdatesHVM"Should be:
"Centos7x8664withUpdatesHVM"