Skip to content

Instantly share code, notes, and snippets.

@jarjuk

jarjuk/aws-must-templates-test-report.md

Last active September 24, 2015 09:10
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jarjuk/9ab1c25d436c4e468f5e to your computer and use it in GitHub Desktop.
Save jarjuk/9ab1c25d436c4e468f5e to your computer and use it in GitHub Desktop.
Download ZIP
Raw
aws-must-templates-test-report.md
Raw
test-suites.md

aws-must-templates - tests

Test suites

smoke - Fails fast if problems with AWS installation - [top]

Creates a S3 bucket, and no EC2 instances.

Stack Parameters and Outputs

AwsMustTestRunnerProperties
  System properties
    property with keys [:stack_id]
      value
        should not eq nil
    property with keys [:suite_id]
      value
        should not eq nil
    property with keys ["Outputs"]
      value
        should not eq nil
    property with keys ["Parameters"]
      value
        should not eq nil

ParameterTest
  Test parameter definition
    Test parameter 'param1' for role 'ParameterTest'
      definition_in_test_suite
        should not eq nil
    Test parameter 'param2' for role 'ParameterTest'
      definition_in_test_suite
        should not eq nil
    Test parameter 'param3' for role 'ParameterTest'
      definition_in_test_suite
        should not eq nil

Stack
  Stack 'smoke' #stack
    #known in test 'Stack''
    Stack parameter 'DummyParameter'
      value
        should not eq nil
    Stack output 'Bucket'
      value
        should not eq nil
    Stack output 'BucketName'
      value
        should not eq nil

Finished in 0.00703 seconds (files took 1.12 seconds to load)
11 examples, 0 failures

Instance Test Reports

suite1 - EC2 instance with S3 read access - [top]

Creates an S3 bucket and two EC2 instances (myInstance, myInstance2) One of the instances (myInstance) is granted Read Access Rights to the Bucket.

To grant Access Rights

  • create a Role
  • create Policy referencing the Role
  • create an InstanceProfile referencing the Role
  • associate the InstanceProfile to the EC2 instance to grant Read Access

Creates SecurityGroup to act as a virtual firewall, and to allow ssh connection to the EC2 instance.

EC2 installation uses UserData script to install

  • AWS Command Line Interface
  • CloudFormation Helper Scripts. These scripts are used to create notification when installation is finished

Stack Parameters and Outputs

Stack
  Stack 'suite1' #stack
    #known in test 'Stack''
    Stack output 'Bucket'
      value
        should not eq nil
    Stack parameter 'InstanceType'
      value
        should not eq nil
    Stack parameter 'KeyName'
      value
        should not eq nil
    Stack parameter 'SSHLocation'
      value
        should not eq nil

Finished in 0.00329 seconds (files took 1.12 seconds to load)
5 examples, 0 failures

Instance Test Reports

suite1-myInstance - [top] - [suite1]

AwsCommandLineInterfaceInstalled
  Command "type aws"
    exit_status
      should eq 0

CloudFormationHelperScriptsInstalled
  Command "type cfn-init"
    exit_status
      should eq 0
  Command "type cfn-signal"
    exit_status
      should eq 0
  Command "type cfn-get-metadata"
    exit_status
      should eq 0
  Command "type cfn-hup"
    exit_status
      should eq 0

Ec2PublicIp
  instance 'myInstance'
    Public IP
      ec2: instanceName=myInstance
        public_ip_address
          should not eql nil

Ec2StatusNormal
  instanceName 'myInstance'
    ec2: instanceName=myInstance
      system_status_not_impaired?
        should eq true
    ec2: instanceName=myInstance
      instance_state_running?
        should eq true

S3ReadAccessAllowed
  Test parameter definition
    Test parameter 'Bucket' for role 'S3ReadAccessAllowed'
      definition_in_test_suite
        should not eq nil
  Test parameter values
    Test parameter 'Bucket' for role 'S3ReadAccessAllowed'
      value
        should not eq nil
  Operating system context
    Aws Commad Line Interface (CLI) is installed
      Command "type aws"
        exit_status
          should eq 0
  When read access to a S3 bucket granted
    Can list S3 buckets
      Command "aws s3 ls"
        exit_status
          should eq 0
    Can list S3 bucket keys
      Command "aws s3 ls s3://suite1-mybucket-2ivcdz2a5nio || aws s3 ls s3://suite1-mybucket-2ivcdz2a5nio --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
        exit_status
          should eq 0
    When an Object exists in S3 bucket
      Can read the Object from a S3 bucket
        Command "aws s3 cp s3://suite1-mybucket-2ivcdz2a5nio/ttest.tmp /tmp/ttest.tmp || aws s3 cp s3://suite1-mybucket-2ivcdz2a5nio/ttest.tmp /tmp/ttest.tmp --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
          exit_status
            should eq 0
      Cannot modify (= delete) the  Object in bucket
        Command "aws s3 rm s3://suite1-mybucket-2ivcdz2a5nio/ttest.tmp --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
          exit_status
            should not eql 0
    Cannot write to bucket
      Create an Object in bucket should fail
        Command "aws s3 cp /etc/hosts  s3://suite1-mybucket-2ivcdz2a5nio/ttest.tmp --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
          exit_status
            should not eql 0
  When bucket does not exists
    Cannot list S3 bucket keys
      Command "aws s3 ls s3://DASKLjwKLJ4534Buckert --region $(aws s3api get-bucket-location --bucket  --output text)"
        exit_status
          should not eq 0

ValidOSVersion
  Operating system codename 'trusty'
    Command "lsb_release --c -s"
      stdout
        should match /trusty/

Finished in 32.12 seconds (files took 1.17 seconds to load)
18 examples, 0 failures

suite1-myInstance2 - [top] - [suite1]

AwsCommandLineInterfaceInstalled
  Command "type aws"
    exit_status
      should eq 0

CloudFormationHelperScriptsInstalled
  Command "type cfn-init"
    exit_status
      should eq 0
  Command "type cfn-signal"
    exit_status
      should eq 0
  Command "type cfn-get-metadata"
    exit_status
      should eq 0
  Command "type cfn-hup"
    exit_status
      should eq 0

Ec2PublicIp
  instance 'myInstance2'
    Public IP
      ec2: instanceName=myInstance2
        public_ip_address
          should not eql nil

Ec2StatusNormal
  instanceName 'myInstance2'
    ec2: instanceName=myInstance2
      system_status_not_impaired?
        should eq true
    ec2: instanceName=myInstance2
      instance_state_running?
        should eq true

S3NoAccess
  When Bucket exists
    #cannot  list Bucket
      Command "aws s3 ls"
        exit_status
          should not eq 0
    When Object exists in Bucket
      #cannot list S3 bucket keys
        Command "aws s3 ls s3://suite1-mybucket-2ivcdz2a5nio --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
          exit_status
            should not eq 0
      #cannot cp S3 bucket object
        Command "aws s3 cp s3://suite1-mybucket-2ivcdz2a5nio/ttest22.tmp /tmp/ttest22.tmp --region $(aws s3api get-bucket-location --bucket suite1-mybucket-2ivcdz2a5nio --output text)"
          exit_status
            should not eq 0

ValidOSVersion
  Operating system codename 'trusty'
    Command "lsb_release --c -s"
      stdout
        should match /trusty/

Finished in 12.07 seconds (files took 1.33 seconds to load)
12 examples, 0 failures

suite2 - VPC with Public and Private Subnets (NAT) - [top]

Creates VPC with Public and Private Subnets (NAT) similar to scenario 2

Subnets:

  • VPC using address range 10.0.0.0/16
  • Public subnet 10.0.0.0/24
    • with internet gateway
    • nat instance myNat
    • front instance myFront1
  • Private Subnet 10.0.1.0/24.
    • back instance myBack1

Defines theree security groups to act as virtual firewalls

  • FrontEndSecurityGroup
  • BackendSecurityGroup:
  • NatSecurityGroup

Defines three instances

  • myNat (based on Amazon vpc-nat instances)
    • defines a public IP
    • private ip from public subnet range
    • route 0.0.0.0/0 to internet gateway
    • allow ping from 0.0.0.0/0
    • allow ssh from 0.0.0.0/0 (as defined parameter SSHLocation)/to Private Subnet
    • allow http from Private Subnet/to 0.0.0.0/0
  • myFront1 (ubuntu)
    • defines a public IP
    • private ip from public subnet range
    • route 0.0.0.0/0 to internet gateway
    • allow ping from 0.0.0.0/0
    • allow ssh from 0.0.0.0/0 (as defined parameter SSHLocation)
  • myBack1 (ubuntu)
    • no public ip assigned
    • private ip from private subnet range
    • route 0.0.0.0/0 to nat instance
    • allow ping from VPC address range
    • allow ssh from public subnet

Stack Parameters and Outputs

AllwaysOk
  Success
    #works

Stack
  Stack 'suite2' #stack
    #known in test 'Stack''
    Stack parameter 'InstanceType'
      value
        should not eq nil
    Stack parameter 'KeyName'
      value
        should not eq nil
    Stack parameter 'SSHLocation'
      value
        should not eq nil
    Stack output 'myFront1'
      value
        should not eq nil
    Stack output 'myNat'
      value
        should not eq nil
    Stack output 'InstanceId1'
      value
        should not eq nil
    Stack output 'InstanceId2'
      value
        should not eq nil
    Stack output 'MyInternetGw'
      value
        should not eq nil

Finished in 0.00601 seconds (files took 1.32 seconds to load)
10 examples, 0 failures

Instance Test Reports

suite2-myNat - [top] - [suite2]

Ec2InstanceType
  instance 'myNat'
    ec2: instanceName=myNat
      instance_type
        should eq "t2.micro"

Ec2PrivateIp
  instance 'myNat'
    ec2: instanceName=myNat, private_ip_address: 10.0.0.233
      #valid cidr 10.0.0.0/24

Ec2PublicIp
  instance 'myNat'
    Public IP
      ec2: instanceName=myNat
        public_ip_address
          should not eql nil

Ec2Routes
  awsRoute:  instanceName=myNat
    implement route {:gateway_id=>"local", :state=>"active", :destination_cidr_block=>"10.0.0.0/16"}
    implement route {:destination_cidr_block=>"0.0.0.0/0", :state=>"active", :gateway_id=>/^igw.*/}

Ec2SecurityGroups
  Security group:  instanceName=myNat
    ingress rules
      WARNING does not validate all ingress rules in test suite
      implement ingress rule: {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement ingress rule: {:ip_protocol=>"tcp", :from_port=>22, :to_port=>22, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement ingress rule: {:ip_protocol=>"tcp", :from_port=>80, :to_port=>80, :ip_ranges=>[{:cidr_ip=>"10.0.1.0/24"}]}
    egress rules
      WARNING does not validate all egress rules in test suite
      implement egress rule:  {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement egress rule:  {:ip_protocol=>"tcp", :from_port=>22, :to_port=>22, :ip_ranges=>[{:cidr_ip=>"10.0.1.0/24"}]}
      implement egress rule:  {:ip_protocol=>"tcp", :from_port=>80, :to_port=>80, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}

Ec2StatusNormal
  instanceName 'myNat'
    ec2: instanceName=myNat
      system_status_not_impaired?
        should eq true
    ec2: instanceName=myNat
      instance_state_running?
        should eq true

NetworkCanPing
  ping  3 times to 'www.google.com', wait response within 20 seconds
    Command "ping www.google.com -W 20 -c 3"
      exit_status
        should eq 0

RespondsToPing
  ping 'myNat'
    ec2: instanceName=myNat
      #reponds within 20 seconds with 3 test counts
      public_dns_name
        should not eql nil

Finished in 8.82 seconds (files took 2.41 seconds to load)
18 examples, 0 failures

suite2-myFront1 - [top] - [suite2]

Ec2InstanceType
  instance 'myFront1'
    ec2: instanceName=myFront1
      instance_type
        should eq "t2.micro"

Ec2PrivateIp
  instance 'myFront1'
    ec2: instanceName=myFront1, private_ip_address: 10.0.0.99
      #valid cidr 10.0.0.0/24

Ec2PublicIp
  instance 'myFront1'
    Public IP
      ec2: instanceName=myFront1, public_ip_address: 52.18.218.246
        public_ip_address
          should eql "52.18.218.246"

Ec2Routes
  awsRoute:  instanceName=myFront1
    implement route {:gateway_id=>"local", :state=>"active", :destination_cidr_block=>"10.0.0.0/16"}
    implement route {:destination_cidr_block=>"0.0.0.0/0", :state=>"active", :gateway_id=>/^igw.*/}

Ec2SecurityGroups
  Security group:  instanceName=myFront1
    ingress rules
      WARNING does not validate all ingress rules in test suite
      implement ingress rule: {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement ingress rule: {:ip_protocol=>"tcp", :from_port=>22, :to_port=>22, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
    egress rules
      WARNING does not validate all egress rules in test suite
      implement egress rule:  {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement egress rule:  {:ip_protocol=>"tcp", :from_port=>22, :to_port=>22, :ip_ranges=>[{:cidr_ip=>"10.0.1.0/24"}]}
      implement egress rule:  {:ip_protocol=>"tcp", :from_port=>443, :to_port=>443, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}

Ec2StatusNormal
  instanceName 'myFront1'
    ec2: instanceName=myFront1
      system_status_not_impaired?
        should eq true
    ec2: instanceName=myFront1
      instance_state_running?
        should eq true

NetworkCanPing
  ping  3 times to 'www.google.com', wait response within 20 seconds
    Command "ping www.google.com -W 20 -c 3"
      exit_status
        should eq 0

RespondsToPing
  ping 'myFront1'
    ec2: instanceName=myFront1
      #reponds within 20 seconds with 3 test counts
      public_dns_name
        should not eql nil

ValidOSVersion
  Operating system codename 'trusty'
    Command "lsb_release --c -s"
      stdout
        should match /trusty/

Finished in 9.91 seconds (files took 2.46 seconds to load)
18 examples, 0 failures

suite2-myBack1 - [top] - [suite2]

Ec2PrivateIp
  instance 'myBack1'
    ec2: instanceName=myBack1, private_ip_address: 10.0.1.246
      #valid cidr 10.0.1.0/24

Ec2PublicIp
  instance 'myBack1'
    Public IP
      ec2: instanceName=myBack1
        public_ip_address
          should eql nil

Ec2Routes
  awsRoute:  instanceName=myBack1
    implement route {:gateway_id=>"local", :state=>"active", :destination_cidr_block=>"10.0.0.0/16"}
    implement route {:destination_cidr_block=>"0.0.0.0/0", :state=>"active", :instance_id=>/^i-.*/}

Ec2SecurityGroups
  Security group:  instanceName=myBack1
    ingress rules
      validate all ingress rules in test suite
      implement ingress rule: {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"10.0.0.0/16"}]}
      implement ingress rule: {:ip_protocol=>"tcp", :from_port=>22, :to_port=>22, :ip_ranges=>[{:cidr_ip=>"10.0.0.0/24"}]}
    egress rules
      validates all egress rules in test suite
      implement egress rule:  {:ip_protocol=>"icmp", :from_port=>-1, :to_port=>-1, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}
      implement egress rule:  {:ip_protocol=>"tcp", :from_port=>80, :to_port=>80, :ip_ranges=>[{:cidr_ip=>"0.0.0.0/0"}]}

Ec2StatusNormal
  instanceName 'myBack1'
    ec2: instanceName=myBack1
      system_status_not_impaired?
        should eq true
    ec2: instanceName=myBack1
      instance_state_running?
        should eq true

NetworkCanPing
  ping  3 times to 'www.google.com', wait response within 20 seconds
    Command "ping www.google.com -W 20 -c 3"
      exit_status
        should eq 0

ValidOSVersion
  Operating system codename 'trusty'
    Command "lsb_release --c -s"
      stdout
        should match /trusty/

Finished in 9.88 seconds (files took 2.25 seconds to load)
14 examples, 0 failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment