Expectations for InSpec controls that are unable to be executed

gistfile1.txt

# UNOFFICAL - FOR DISCUSSION ONLY

# Test inspec under conditions in which it is unable to perform checks.

# NOTE: many of these mention doing "negative tests"  Here's an example:
# describe file('/this/does/not/exist') do
#   its('contents') { should_not include 'bananas'}
# end
#
# Depending on the particular resource, in InSpec 2.3.13, that test may fail, skip, or pass.
# The idea behind this initiative is to always throw an UnAble exception, and then give the
# calling system latitude in how to handle that. By default, it would always be a failure.

# Suite A - Resources and Basic Ability

# Case A-01
# When a profile contains a file resource,
#  and the file does not exist,
#  and the control performs a negative content test
#  then the resource generates an UnAble exception subclass
#  then the control fails with the message that it does not exist

# Case A-02
# When a profile contains a shadow resource,    # <--- or any other resource that uses file(), perhaps several layers down
#  and the file does not exist,
#  and the control performs a negative user check
#  then the resource generates an UnAble exception subclass
#  then the control fails with the message that the underlying file does not exist

# Case A-03
# When a profile contains a command resource,
#  and the named command does not exist on the system,
#  and the control performs a negative stdout test,
#  then the resource generates an UnAble exception subclass
#  then the control should fail with the message that the underlying command is missing

# Case A-04
# When a profile contains a http resource,    # <--- or any other resource that uses command(), perhaps several layers down
#  and the curl is not installed on the system
#  and the control performs a negative body check
#  then the resource generates an UnAble exception subclass
#  then the control fails with the message that curl is not installed

# Case A-05
# When a database control attempts to connect to a database,
#  and the network connection times out
#  and the control contains a negative test,
#  then the resource generates an UnAble exception subclass
#  then the control should fail with a timeout message

# Case A-06
# When a cloud-based resource is used to locate a cloud object by id
#  and the search is a miss
#  and the control contains any test other than an existance check
#  then the resource generates an UnAble exception subclass
#  then the control should fail with information about the missing cloud object

# Suite B - Privilege

# Case B-01
# When a resource is used that would require sudo (???)  # <-- aside from manual inspection, how would we know?
#  and the target is run without elevated permissions
#  then the resource generates an UnAble exception subclass
#  then the control should fail with the message that the sudo is required to use the control

# Case B-02
# When a resource is used that would write to the target disk
#  and that is not permitted (? how so? inspec config?  attempt and fail?)
#  then the resource generates an UnAble exception subclass
#  then the control should fail with the message that write access is needed for that control

# Case B-03
# When a cloud-based resource is used
#  and the credentials are correct for basic authentication
#  and the credentials do not allow a specific needed operation
#  then the resource generates an UnAble exception subclass
#  then the control should fail with information about the needed permission

# Suite C - Conditionalized Handling

# Case C-01
# When a control is unable to be executed
#  and there is no other unable-handling condition in place
#  then the resource generates an UnAble exception subclass
#  then the control should fail

# Case C-02
# When a control is unable to be executed
#  and there is a skip_on_unable (DSL statement? block?)
#  then the resource generates an UnAble exception subclass
#  then the control should skip

# Case C-03
# When profile W wraps profile D
#  and profile D contains a control C that cannot be executed
#  and profile W overrides the control C, adding the DSL 'skip_on_unable'
#  and profile W is executed
#  then the resource generates an UnAble exception subclass
#  then the control will be skipped

# Case C-04
# When profile W wraps profile D
#  and profile D contains a control C that cannot be executed
#  and control C contains the DSL statement 'skip_on_unable'
#  and profile W overrides the control, adding the DSL 'fail_on_unable'
#  and profile W is executed
#  then the resource generates an UnAble exception subclass
#  then the control will fail

#----------------------------------

# Future work:
#   Unable-type exceptions lay the groundwork for intercepting network access attempts
#     especially if a block approach is used
#     foundation for airgap support
#   Unable-type exceptions lay the groundwork for intercepting filesystem access attempts
#     foundation for selinux/etc support
#     foundation for --no-write option
#     foundation for auditing fs access