Skip to content

Instantly share code, notes, and snippets.

@ckabalan
Last active September 14, 2019 18:52
Show Gist options
  • Save ckabalan/84197739d839ff6c6cb42520331f717a to your computer and use it in GitHub Desktop.
Save ckabalan/84197739d839ff6c6cb42520331f717a to your computer and use it in GitHub Desktop.
CloudFormation Package Issue
> git clone https://github.com/passthesecret/serverless.git
Cloning into 'serverless'...
remote: Enumerating objects: 75, done.
remote: Counting objects: 100% (75/75), done.
remote: Compressing objects: 100% (54/54), done.
remote: Total 75 (delta 29), reused 62 (delta 16), pack-reused 0
Unpacking objects: 100% (75/75), done.
> cd serverless
> git checkout 4033970
[ ... SNIP ... ]
HEAD is now at 4033970 Remove Container Use
> ll
total 40K
drwxrwxr-x 6 ckabalan ckabalan 4.0K Sep 14 18:28 ./
drwxr-xr-x 24 ckabalan ckabalan 4.0K Sep 14 18:28 ../
drwxrwxr-x 8 ckabalan ckabalan 4.0K Sep 14 18:28 .git/
drwxrwxr-x 5 ckabalan ckabalan 4.0K Sep 14 18:28 deployment/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:28 passthesecret/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:28 tests/
-rw-rw-r-- 1 ckabalan ckabalan 3.7K Sep 14 18:28 .gitignore
-rw-rw-r-- 1 ckabalan ckabalan 4.0K Sep 14 18:28 .gitlab-ci.yml
-rw-rw-r-- 1 ckabalan ckabalan 3.9K Sep 14 18:28 swagger.yaml
-rw-rw-r-- 1 ckabalan ckabalan 2.1K Sep 14 18:28 template.yaml
> sam build
Building resource 'CreateSecret'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Building resource 'GetSecret'
Running PythonPipBuilder:ResolveDependencies
Running PythonPipBuilder:CopySource
Build Succeeded
Built Artifacts : .aws-sam/build
Built Template : .aws-sam/build/template.yaml
Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Package: sam package --s3-bucket <yourbucket>
> ll
total 44K
drwxrwxr-x 7 ckabalan ckabalan 4.0K Sep 14 18:29 ./
drwxr-xr-x 24 ckabalan ckabalan 4.0K Sep 14 18:29 ../
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 .aws-sam/
drwxrwxr-x 8 ckabalan ckabalan 4.0K Sep 14 18:28 .git/
drwxrwxr-x 5 ckabalan ckabalan 4.0K Sep 14 18:28 deployment/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:28 passthesecret/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:28 tests/
-rw-rw-r-- 1 ckabalan ckabalan 3.7K Sep 14 18:28 .gitignore
-rw-rw-r-- 1 ckabalan ckabalan 4.0K Sep 14 18:28 .gitlab-ci.yml
-rw-rw-r-- 1 ckabalan ckabalan 3.9K Sep 14 18:28 swagger.yaml
-rw-rw-r-- 1 ckabalan ckabalan 2.1K Sep 14 18:28 template.yaml
##### Note all the prerequisites and .dist-info
> ll .aws-sam/build/CreateSecret
total 996K
drwxrwxr-x 31 ckabalan ckabalan 4.0K Sep 14 18:29 ./
drwxr-xr-x 4 ckabalan ckabalan 4.0K Sep 14 18:29 ../
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 .libs_cffi_backend/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 asn1crypto/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 asn1crypto-0.24.0.dist-info/
drwxrwxr-x 9 ckabalan ckabalan 4.0K Sep 14 18:29 boto3/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 boto3-1.9.225.dist-info/
drwxrwxr-x 5 ckabalan ckabalan 4.0K Sep 14 18:29 botocore/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 botocore-1.12.225.dist-info/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 cffi/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 cffi-1.12.3.dist-info/
drwxrwxr-x 4 ckabalan ckabalan 4.0K Sep 14 18:29 cryptography/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 cryptography-2.7.dist-info/
drwxrwxr-x 5 ckabalan ckabalan 4.0K Sep 14 18:29 dateutil/
drwxrwxr-x 8 ckabalan ckabalan 4.0K Sep 14 18:29 docutils/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 docutils-0.15.2.data/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 docutils-0.15.2.dist-info/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 jmespath/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 jmespath-0.9.4.data/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 jmespath-0.9.4.dist-info/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 ptscore/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 ptscore-0.1.7.dist-info/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 pycparser/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 pycparser-2.19.dist-info/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 python_dateutil-2.8.0.dist-info/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 s3transfer/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 s3transfer-0.2.1.dist-info/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 six-1.12.0.dist-info/
drwxrwxr-x 3 ckabalan ckabalan 4.0K Sep 14 18:29 tests/
drwxrwxr-x 5 ckabalan ckabalan 4.0K Sep 14 18:29 urllib3/
drwxrwxr-x 2 ckabalan ckabalan 4.0K Sep 14 18:29 urllib3-1.25.3.dist-info/
-rw-rw-r-- 1 ckabalan ckabalan 0 Sep 14 18:28 __init__.py
-rw-rw-r-- 1 ckabalan ckabalan 830K Sep 14 18:29 _cffi_backend.cpython-37m-x86_64-linux-gnu.so
-rw-rw-r-- 1 ckabalan ckabalan 1.8K Sep 14 18:28 passthesecret.py
-rw-rw-r-- 1 ckabalan ckabalan 217 Sep 14 18:28 requirements.txt
-rw-rw-r-- 1 ckabalan ckabalan 32K Sep 14 18:29 six.py
##### Note small file size uploaded to S3
> aws cloudformation package --template-file template.yaml --s3-bucket passthesecret-ci-bucket --output-template-file packaged.yaml
Uploading to f8ad706f79bd5258a840b9ec9b73b755 1096 / 1096.0 (100.00%)
Successfully packaged artifacts and wrote output template to file packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/ckabalan/serverless/packaged.yaml --stack-name <YOUR STACK NAME>
##### Download the file uploaded to S3 to inspect it
> aws s3 cp s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755 f8ad706f79bd5258a840b9ec9b73b755
download: s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755 to ./f8ad706f79bd5258a840b9ec9b73b755
##### Note there are no prerequisites, explaining the small file size
> unzip -l f8ad706f79bd5258a840b9ec9b73b755
Archive: f8ad706f79bd5258a840b9ec9b73b755
Length Date Time Name
--------- ---------- ----- ----
1769 2019-09-14 18:28 passthesecret.py
217 2019-09-14 18:28 requirements.txt
0 2019-09-14 18:28 __init__.py
--------- -------
1986 3 files
##### Clean up
> aws s3 rm s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755
delete: s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755
> rm f8ad706f79bd5258a840b9ec9b73b755
##### Lets try the same thing with "aws cloudformation" replaced with "sam"
##### Note the small file size again
> sam package --template-file template.yaml --s3-bucket passthesecret-ci-bucket --output-template-file packaged.yaml
Uploading to f8ad706f79bd5258a840b9ec9b73b755 1096 / 1096.0 (100.00%)
Successfully packaged artifacts and wrote output template to file packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/ckabalan/serverless/packaged.yaml --stack-name <YOUR STACK NAME>
##### Download the file uploaded to S3 to inspect it
> aws s3 cp s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755 f8ad706f79bd5258a840b9ec9b73b755
download: s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755 to ./f8ad706f79bd5258a840b9ec9b73b755
##### Note there are no prerequisites, explaining the small file size. Same behavior as "aws cloudformation" command
> unzip -l f8ad706f79bd5258a840b9ec9b73b755
Archive: f8ad706f79bd5258a840b9ec9b73b755
Length Date Time Name
--------- ---------- ----- ----
1769 2019-09-14 18:28 passthesecret.py
217 2019-09-14 18:28 requirements.txt
0 2019-09-14 18:28 __init__.py
--------- -------
1986 3 files
##### Clean up
> aws s3 rm s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755
delete: s3://passthesecret-ci-bucket/f8ad706f79bd5258a840b9ec9b73b755
> rm f8ad706f79bd5258a840b9ec9b73b755
##### Now lets try without specifying --template-file. It fails because it is a required argument.
> aws cloudformation package --s3-bucket passthesecret-ci-bucket --output-template-file packaged.yaml
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: the following arguments are required: --template-file
##### Lets try the same failed command above, except "sam" instead of "aws cloudformation"
##### Note the larger filesize
> sam package --s3-bucket passthesecret-ci-bucket --output-template-file packaged.yaml
Uploading to eb957b31bf91e14b837c4312ac873a3a 9802700 / 9802700.0 (100.00%)
Successfully packaged artifacts and wrote output template to file packaged.yaml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/ckabalan/serverless/packaged.yaml --stack-name <YOUR STACK NAME>
##### Download the file to inspect it
> aws s3 cp s3://passthesecret-ci-bucket/eb957b31bf91e14b837c4312ac873a3a eb957b31bf91e14b837c4312ac873a3a
download: s3://passthesecret-ci-bucket/eb957b31bf91e14b837c4312ac873a3a to ./eb957b31bf91e14b837c4312ac873a3a
##### Note all the prerequisites are here
> unzip -l eb957b31bf91e14b837c4312ac873a3a
Archive: eb957b31bf91e14b837c4312ac873a3a
Length Date Time Name
--------- ---------- ----- ----
1769 2019-09-14 18:28 passthesecret.py
32452 2019-09-14 18:29 six.py
217 2019-09-14 18:28 requirements.txt
0 2019-09-14 18:28 __init__.py
849744 2019-09-14 18:29 _cffi_backend.cpython-37m-x86_64-linux-gnu.so
4 2019-09-14 18:29 six-1.12.0.dist-info/top_level.txt
110 2019-09-14 18:29 six-1.12.0.dist-info/WHEEL
435 2019-09-14 18:29 six-1.12.0.dist-info/RECORD
1066 2019-09-14 18:29 six-1.12.0.dist-info/LICENSE
1940 2019-09-14 18:29 six-1.12.0.dist-info/METADATA
[ ... SNIP ~1300 LINES ... ]
9 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/top_level.txt
110 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/WHEEL
2044 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/RECORD
1 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/zip-safe
2889 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/LICENSE
7647 2019-09-14 18:29 python_dateutil-2.8.0.dist-info/METADATA
--------- -------
51866096 1311 files
##### Show versions
> aws --version
aws-cli/1.16.238 Python/3.6.8 Linux/4.15.0-62-generic botocore/1.12.228
> sam --version
SAM CLI, version 0.21.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment