This guideline shows how to identify test
ideas from statecharts and other design structures that consist mainly of nodes connected by arcs and that show
something of the possible control flows of a program. The main goal of this testing is to traverse every arc in some
test. If you've never exercised an arc, why do you think it will work when a customer does?
Consider this statechart:
Fig1: HVAC Statechart
Here's a first list of test ideas:
Idle state receives Too Hot event
Idle state receives Too Cool event
Cooling/Startup state receives Compressor Running event
Cooling/Ready state receives Fan Running event
Cooling/Running state receives OK event
Cooling/Running state receives Failure event
Failure state receives Failure Cleared event
Heating state receives OK event
Heating state receives Failure event
These test ideas could all be exercised in a single test, or you could create several tests that each exercise a few.
As with all test design, strive for a balance between the ease of implementation of many simple tests and the
additional defect-finding power of complex tests. (See "test design using the list" in the Concept: Test
Ideas List page.) If you have use case scenarios that describe certain paths through the statechart, you should
favor tests that take those paths.
In any case, the tests should check that all actions required by the statechart actually take place. For example, is
the alarm started on entry to the Failure state, then stopped upon exit?
The test should also check that the transition leads to the correct next state. That can be a difficult problem when
the states are invisible from the outside. The only way to detect an incorrect state is to inject some sequence of
events that leads to incorrect output. More precisely, you would need to construct a follow-on sequence of events whose
externally-visible results for the correct state differ from those that the same sequence would provoke from
each possible incorrect state.
In the example above, how would you know that the Failure Cleared event in the Failure state correctly led to the Idle
state, instead of staying in the Failure state? You might trust that the stopping of the Alarm meant that transition
had been made, but it might be better to check by lowering the temperature enough to make the heater start or raising
it enough to turn on cooling. If something happens, you're more confident that the transition was correct. If nothing
happens, it's likely the device stayed in the Failure state.
At the very least, determining whether the resulting state is correct complicates test design. It is often better to
make the state machine explicit and make its states visible to the tests.
Other statechart constructs
Statecharts consist of more than arcs and arrows. Here is a list of statechart constructs and the effect they have on
the test idea list.
Event actions, entry actions, and exit actions
These do not generate test ideas per se. Rather, the tests should check that the actions behave as specified. If the
actions represent substantial programs, those programs must be tested. The test ideas for the programs might be
combined with test ideas from the statechart, but it's probably more manageable to separate them. Make the decision
based on the effort involved and on your suspicion that there might be interactions between events. That is, if a
particular action on one arc cannot possibly share data with an action on another arc, there is no reason to exercise
the two actions in the same test (as you would if they were part of the same path through a statechart test).
Guard conditions are boolean expressions. The test ideas for guard conditions are derived as described in Guideline: Test Ideas for Booleans and Boundaries.
In the example above, the Too Cool transition from the Idle state is guarded with [restart time >= 5 mins]. That
leads to two separate test ideas:
Idle state receives Too Cool event when restart time is five minutes (transition taken)
Idle state receives Too Cool event when restart time is just less than five minutes (transition blocked)
In both cases, any test that uses the test idea should check that the correct state is reached.
An internal transition adds the same sort of ideas to a test idea list as an external transition does. It's merely that
the next state is the same as the original state. It would be prudent to set up the test such that the state's entry
and exit actions would cause an observable effect if they were incorrectly triggered.
When constructing tests, set them up such that entry and exit events of the composite state have observable effects.
You want to notice if they're skipped.
Testing of concurrency falls outside of the scope of developer testing.
If you suspect an event might be handled differently depending on whether it was deferred and queued rather than
generated while the program was actually in the receiving state, you might test those two cases.
If the event in the receiving state has a guard condition, consider the ramifications of changes to the condition's
variables between the time the event is generated and the time it is received.
If more than one state can handle a deferred event, consider testing deferral to each of the possible receiving states.
Perhaps the implementation assumes that the "obvious" state will handle the event.
Here is an example of a history state:
Fig2: History State Example
The transition into the history state represents three real transitions, and thus three test ideas:
BackupUp event in Command state leads to Collecting state
BackupUp event in Command state leads to Copying state
BackupUp event in Command state leads to CleaningUp state
Chain states do not seem to have any implications for test design, except that they introduce more actions that need to
The preceding discussion focuses on checking whether the implementation matches the design. But the design might also
be wrong. While examining the design to find test ideas, also check for two types of problems:
Missing events. The statechart shows a state's response to events that the designer anticipated could arrive
in that state. It's not unknown for designers to overlook events. For example, in this statechart (repeated from
the top of the page), perhaps the designer forgot that a failure can occur in the Ready substate of Cooling, not just
when the fan is Running.
Fig3: HVAC Statechart
For this reason, it's wise to ask, for each state, whether any of the events that apply to other states might apply to
this one. If you discover that one does, correct your design.
Incomplete or missing guard conditions. Similarly, perhaps guard conditions on one transition will suggest guard
conditions on others. For example, the above statechart takes care not to restart the heater too often, but there is no
such restriction on the cooling system. Should there be?
It is also possible that variables used on one guard condition will suggest that other guard conditions are too simple.
Testing each arc in a graph is by no means complete testing. For example, suppose the start state initializes a
variable to 0, state Setter sets it to 5, and state Divider divides it into 100 (100/variable). If there's a path from
the start state to Divider that does not pass through Setter, you have a divide-by-zero exception. If the statechart
has many states, simply exercising each arc might miss that path.
Except for very simple statecharts, testing every path is infeasible. In practice, tests that are complex and
correspond to use case scenarios are often sufficient. If you desire stronger tests, consider requiring a path from
each state where a datum is given a value to each state that uses it.