Now it is time for the main event: actually writing the test
case steps and specifying test data. This is a task that you can
expect to take ten to forty-five minutes for each test case. That
might work out to approximately ten test cases in a typical work
day. So, you must be selective to get the most value in
return for your limited available time.
Focus on the test cases that seem most in need of additional
detail. For example, select system test cases that cover:
High priority use cases or features
Software components that are currently available for testing
(rather than specifying tests on components that cannot
actually be tested yet)
Features that must work properly before other features
can be exercised (e.g., if login does not work, you cannot test
anything that requires a logged in user)
Features that are needed for product demos or screenshots
Requirements that need to be made more clear
Each test case should be simple enough to clearly succeed or
fail, with little or no gray area in between. Ideally, the steps
of a test case are a simple sequence: set up the test
situation, exercise the system with specific test inputs, verify
the correctness of the system outputs. You may use programming
constructs such as if-statements or loops, if needed.
Systems that are highly testable tend to have a large number
of simple test cases that follow the set-up-exercise-verify
pattern. For those test cases, a one-column format can clearly
express the needed steps. However, not all test cases are so
simple. Sometimes it is impractical to test one requirement at a
time. Instead, some system test cases may be longer scenarios
that exercise several requirements and verify correctness at each
step. For those test cases, a two-column format can prove useful.
In the one-column format, each step is a brief verb phrase
that describes the action that the tester should take. For
example, "enter username," "enter password," "click 'Login',"
"see Welcome page," and "verify that greeting has correct
username" are all steps. Verification of expected outputs are
written using the verbs "see" and "verify." If multiple inputs
are needed, or multiple outputs must be verified, one-column test
cases will simply have more steps.
In the two-column format, each test case step has two parts: a
test input, and an expected output:
The Test Input is a verb phrase describing what the tester
should do in that step.
The Expected Output is a noun phrase describing all the
output that the tester should observe at that step.
You may notice that the two formats for test cases mirror the
two formats for use cases. The difference is that use cases are
a form of requirements, whereas test cases deal with more details
of the implemented system. Use cases focus mainly on the user's
tasks and how the system supports those tasks, while specifying
as few implementation details as possible. A major advantage of
use cases is that they are simple enough to be read by actual
users who can help validate requirements. In contrast, test
cases should more technical documents with enough implementation
detail to allow any member of the development team to carry out a
test exactly the same way.
If you have written use cases, they can be copied and pasted
as a good starting point for test cases. When leveraging use
cases in this way, make sure to add enough detail to make the
test reliably repeatable.
If you only have one test input value for a given test case,
then you could write that test data value directly into the step
where it is used. However, many test cases will have a set of
test data values which must all be used to adequately cover all
possible inputs. We encourage you to define and use test input
variables. Each variable is defined with a set of its selected
values, and then it is used in test case steps just as you would
use a variable in a programming language. When carrying out the
tests, the tester should repeat each test case with each possible
combination of test variable values, or as many as practical.
Carefully selecting test data is as important as defining the
steps of the test case. The concepts of boundary conditions and
equivalence partitions are key to good test data selection.
Try these steps to select test data:
Determine the set of all input values that can possibly be
entered for a given input parameter. For example, the age of a
person might be entered as any integer.
Define the boundary between valid and invalid input values.
For example, negative ages are nonsense. You might also check
for clearly unreasonable inputs. For example, an age entered as
200 is much more likely to be a typo than a user who is actually
two-hundred years old.
Review the requirements and find boundaries in the valid
range that should cause the system to behave in different ways.
For example, the system might treat minors differently than
adults, so the boundary would be age 18.
Now you have a set of equivalence partitions: sets of values
that the system should treat uniformly. For example, all minors
are treated one way, and all adults are treated another way.
Double check the requirements to make sure that you have not
missed a partition division, e.g., not all adults are old enough
to drink alcohol in the U.S.
Choose one input value somewhere in the middle of each
equivalence partition (e.g., -5, 12, and 44), one directly on
each boundary (e.g., 0 and 18), and one on each side of each
boundary (e.g., 1, 17, and 19). Test data vales that are
expected to cause errors (e.g., -5) should be tested in separate
robustness test cases.
In functional correctness test cases, make sure that you
have inputs that will force the system to generate each possible
type of response to valid input. And, in robustness test cases,
make sure to force the system to generate each relevant error
Recall that one of the advantages of writing test cases is
that it forces you to clearly think through the requirements.
Capture your insights by writing notes and questions as you go.
If a test case step exposes an unclear requirement, make a note
of it in the appropriate part of the system requirements
Step Six: Evaluate Test Cases
A suite of system test cases can find many defects, but still
leave many other critical defects undetected. One clear way to
guard against undetected defects is to increase the coverage of
your test suite.
While a suite of unit tests might be evaluated in terms of its
implementation coverage, a suite of system test cases should
instead be evaluated in terms of specification coverage.
Implementation coverage measures the percentage of lines of code
that are executed by the unit test cases. If there is a line of
code that is never executed, then there could be an undetected
defect on that line. Specification coverage measures the
percentage of written requirements that the system test suite
covers. If there is a requirement that is not tested by any
system test case, then you are not assured that the requirement
has been satisfied.
You can evaluate the coverage of your system tests on two
levels. First, the test suite itself is an organized table of
contents for the test cases that can make it easy to notice parts
of the system that are not being tested. Second, within an
individual test case, the set of possible input values should
cover all input value equivalence partitions for each parameter.
This white paper laid out the steps needed to quickly create a
system test suite and test cases using the ReadySET Pro templates.
The keys to effectively system testing are to:
Set explicit quality goals that are appropriate for the
current release and understand where your test plan fits in your
overall QA plan.
Take a breadth-first approach by first mapping out a test
suite with good coverage, and then prioritizing your work on
specifying the test cases themselves.
Write system test cases in enough detail that any member of
the development team could carry out the testing to see the same
results, and choose test input values that cover all equivalence
Evaluate the system test suite and test cases to improve
coverage of the system requirements.
ReadySET Pro provides valuable help for planning your system
testing by giving you templates that include reusable content and
set good examples for you to follow. Both of these advantages
give you a big head start on your own test plans. ReadySET Pro
users typically save at least three hours by using the test case
suite and test case templates alone. See how these savings, and
the savings from other templates, add up to days or weeks of
project time by trying the ROI