In this third post in this series on the fundamentals of software testing, I finish the topic of which values to test as part of your Verification Testing which I started in my previous post on using Boundary-Value Analysis.
Even if you can test all possible values of what you are testing (also known as the system under test), you do not need to waste the time and effort. Instead, you can use the concept of Equivalence Class Partitioning. This technique allows you to test a single value from each identified equivalence class instead of all possible values and still achieve the same test coverage.
What is Equivalence Class Partitioning? Basically it is dividing all the possible input values (or conditions) of your system under test into partitions (or classes) that are logically the same. You will use both the Acceptance Criteria (or requirements and/or specifications) and the internal logic (i.e. the "code") of your system under test to do this partitioning.
🤓 Looking at the internals of your system under test to drive externally-focused behavioral (i.e. "functional") testing is called Gray-Box Testing.
With Equivalence Class Partitioning you will also use the concept of Boundary-Value Analysis to identify each of your input value boundaries: the lowest-supported value and the highest-supported value. Each of the boundary values becomes their own equivalence class.
Consider the example presented in my previous post on Boundary-Value Analysis. Your Product Owner provides you the following Acceptance Criteria for your software system: The system must accept and produce the correct behavior (which you can assume for this example is already well-defined elsewhere) for values between 1 and 100 inclusively.
Applying Equivalence Class Partitioning in combination with Boundary-Value Analysis, you would have 3 equivalence classes for the supported values:
- The lowest boundary value of 1
- The highest boundary value of 100
- The set of values between but not including the boundary values i.e. [2..99]
Thus, you must test this system with just three values to verify that it works as expected for the supported value:
- The value 1
- The value 100
- Any value from 2 to 99 inclusively
Testing just these 3 values is a lot more efficient than testing all the values from 1 to 100 inclusively.
Now consider a different example where you have a system under test that has internal "code" that handles odd and even numbers differently (for example, if odd do this, else do that). Even if this system under test produces the same exact behavior for odd and even numbers, these would still be different equivalence classes and both must be tested separately.
Finally, sampling (sometimes called "fuzzing") is a great way to test different input values for your system under test, but you should only ever sample a specific value from within an equivalence class (for example, sampling a value from your equivalence class of numbers from [2..99] inclusively) and not across equivalence classes (for example sampling a value from all your supported input values of [1..100] inclusively). The reason for this is that you would then not be testing all of the cases for your system under test allowing bugs to go uncaught.
⏮️ Fundamentals of Testing 2: What Values to Test - Boundary-Value Analysis