TESTING
SOFTWARE QA
RESOURCES
Notes taken from Boris Beizer books:

Define a set of covering input sequences that get back to the initial state when starting from the initial state

For each step in each input sequence, define the expected next state, the expected transition, and the expected outcome.

A set of tests, then, consists of three sets of sequences:

  • Input sequences
  • Corresponding transitions or next-state names
  • Outcome sequences
  • Simply identifying the factors that contribute to the state, calculating the total number of states, and comparing this number to the designer's notion catches some bugs

    Insisting on a justification for all supposedly dead, unreachable, and impossible states and transitions catches a few more bugs

    Insisting on an explicit specification of the transition and output for every combination of input and state catches many more bugs

    A set of input sequences that provide coverage of all state graph nodes and links is a mandatory minimum requirement

    In executing state tests, it is essential that means be provided (e.g. instrumentation software) to record the sequence of states (e.g. transitions) resulting from the input sequence and not just the outputs that result from the input sequence.

    More than most test methods, state-graph tests yield their biggest payoffs during the design of the tests rather than during the running thereof.

    What to model:

  • Any processing where the output is based on the occurrence of one or more sequences of events, such as detection of specified input sequences, sequential format validation, and other situations in which the order of inputs is important
  • Most protocols between systems, between humans and machines, between modules within a system
  • Device handlers such as tape handlers, disc handlers, and the like, that involve complicated retry and recovery procedures if the action depends on the state.
  • Transaction flows where the transactions are such that they can stay in the system indefinitely. For example, online users, tasks in a multi-programming environment
  • High-level control functions within an operating system. Transitions between user states, supervisor's states, and so on. Security handling of records, permission for read/write/modify privileges, priority interrupts and transitions between interrupt states and levels, recovery issues and the safety state of records and/or processes with respect to recording recovery data
  • The behaviour of the system with respect to resource management and what it will do when various levels of resource utilization are reached. Any control function that involves responses to thresholds where the system's action depends not just on the threshold value, but also on the direction in which the threshold is crossed. This is a normal approach to control functions. A threshold passage in one direction stimulates a recovery function, but that recovery function is not suspended until a second, lower threshold is passed going the other way.
  • Whenever a function is directly and explicitly implemented as one or more state-transition tables.