"Hierarchical GUI Test Case Generation Using Automated Planning" "Plan Generation for GUI Testing"

1. GUI Testing
Approaches
Will Shen
2004/12/21

2. References
 Hierarchical GUI Test Case Generation
Using Automated Planning
 Atif M. Memon, Student Member, IEEE, Martha E.
Pollack, and Mary Lou Soffa, Member, IEEE
 Plan Generation for GUI Testing
 Atif M. Memon and Martha E. Pollack and Mary
Lou Soffa Dept. of Computer Science University of
Pittsburgh Pittsburgh, PA 15260 USA fatif, pollack,
soffag@cs.pitt.edu

3.  Coverage Criteria for GUI Testing
 Atif M. Memon Dept. of Computer Science
University of Pittsburgh Pittsburgh, PA 15260
atif@cs.pitt.edu
 Mary Lou Soffa Dept. of Computer Science
University of Pittsburgh Pittsburgh, PA 15260
soffa@cs.pitt.edu
 Martha E. Pollack Dept. of Electrical Engineering
and Computer Science, University of Michigan Ann
Arbor, MI 48109pollackm@eecs.umich.edu

4. Outline
1. Introduction
2. Overview of PATHS
3. Coverage Criteria for GUI Testing
4. Conclusions

5. 1. Introduction
 Testing GUI is difficult:
 The space of possible interactions with a GUI is
enormous.
 In that each sequence of GUI commands can result in a
different state
 A GUI command may need to be evaluatedion all of these
states.
 Determining the coverage of a set of test cases.
 No only how much the code is tested, but in how many
different possible states of the software each piece of code
is tested.

6.  An important aspect of GUI testing is
verification of its state at each step of test case
execution.
 The execution of the test case must be terminated as
soon as an error is detected.
 Regression testing presents special challenges for
GUIs.
 The input-output mapping does not remain constant
across successive versions of the software.

7. 2. Overview of PATHS
 Planning Assited Tester for grapHical user
interface Systems.
 A new approach to automatic testing of GUIs
that builds on AI planning techniques.
 Given a specification of initial and goal states for
a GUI, a planner is used to generate sequences
of GUI actions that lead from the initial state to
the goal state.

8. 2.1 The Example GUI
Microsoft WordPad

9. (a) the Initial States
(b) The Goal State

10. 2.2 Test case generation process
 Two phases
 Setup phase
1. PATHS creates a hierarchical model of the GUI and
returns a list of operators from the model to the test
designer
2. By using knowledge of the GUI, the test designer then
defines the preconditions and effects of the operators in
a simple language provided by the planning system.

11.  Plan-generation
1. The test designer describes scenarios (tasks) by defining
a set of initial and goal states for test case generation.
2. PATHS generates a test suite for the scenarios.

13. 2.3 Deriving GUI operators
 2.3.1 Events
 2.3.2 Operators
 2.3.3 Operator-Event mapping
 2.3.4 The Step1 of example GUI
 2.3.5 The Step2 of EDIT_CUT
 2.3.6 The Step3 initial state and goal state
 2.3.7 The Step4 generate test case

14. 2.3.1 Events
 Three classes of GUI event
 Unrestricted-focus events open GUI windows that
do not restrict the user’s focus
 Restricted-focus events open GUI windows that
have the special property that once invoked, they
monopolize the GUI interaction.
 System-interaction events interact with the
underlying software to perform some action.

15. 2.3.2 Operators
 The setup phase starts creating a list of a
operators to be used during planning.
 Exploiting the GUI structure to derive
hierarchical operators that are decomposed
during planning.
 System-Interaction Operators
 Abstract Operators

16. System-Interaction Operators
 To represent sequences of GUI actions that a
user might perform to eventually interact with
the underlying software.
 A sequence of zero or more unrestricted-focus
events, followed by a system-interaction event.
 Example:
 Edit_Cut = <Edit, Cut>
 Edit_Paste = <Edit, Paste>

17. Abstract Operators
 Created from the restricted-focus events, which
contain two parts:
 The prefix of an abstract operator is the sequence of
unrestricted-focus events that lead to restricted-
focus event.
 The suffix of an abstract operator represents the
restricted-focus user interaction.

18. 2.3.3 Operator-Event mapping
 In order to keep a correspondence between the
original GUI events and these high-level
operators.

19. 2.3.4 The Step1 of example GUI
(a) Original GUI Events
(b) Planning operators derived by PATHS

20. 2.3.5 The Step2 of EDIT_CUT

21. 2.3.6 The Step3 initial state and goal
state

22. 2.3.7 The Step4 generate test case
High level plan that must be decomposition
decomposition decomposition

25. An alternative decomposition

26. A new test case

27. 3. Coverage Criteria for GUI Testing
 3.1 What is Coverage Criteria
 3.2 Event-flow Graphs
 3.3 Integration Tree
 3.4 Intra-component Coverage Criteria
 3.5 Inter-component Coverage Criteria

28. 3.1 What is Coverage Criteria
 Coverage criteria are sets of rules to help
determine whether a test suite has adequately
tested a program and to guide the testing
process.
 Important rules that provide an objective
measure of test quality.

29.  This paper define a new class of coverage
criteria called event-based coverage criteria to
determine the adequacy of tested event
sequences, focus on GUIs.
 The key idea is to define the coverage of a test
suite in terms of GUI events and their
interactions.

30.  A GUI component is represented by a new
structure called an event-flow graph that
identifier events within a component.
 The interactions among GUI components are
captured by a representation called the
integration tree.

31.  Intra-component coverage criteria for events
within a component
 event, event-interaction and length-n event-sequence
coverage.
 Inter-component coverage criteria for events
among components.
 invocation, invocation-termination and length-n
event-sequence coverage

32. 3.2 Event-flow Graphs

33. A part of the Main* component of MS
B WordPad
I I I
*
We assume that all GUIs have a Main component, that is presented to the
user when the GUI is first invoked.

34. 3.3 Integration Tree

35. An integration tree for a part of MS WordPad

36. 3.4 Intra-component Coverage
Criteria
 3.4.1 Event Coverage
 3.4.2 Event-interaction Coverage
 3.4.3 Length-n Event-sequence Coverage

37. 3.4.1 Event Coverage
Event-sequence:

38. 3.4.2 Event-interaction Coverage

39. 3.4.3 Length-n Event-sequence
Coverage

40. 3.5 Inter-component Coverage
Criteria
 3.5.1 Invocation Coverage
 3.5.2 Invocation-termination Coverage
 3.5.3 Inter-component Length-n Event-
sequence Coverage

41. 3.5.1 Invocation Coverage

42. 3.5.2 Invocation-termination
Coverage

43. 3.5.3 Inter-component Length-n
Event-sequence Coverage

44. 4. Conclusion
 Automatic testing of GUIs that builds on AI
planning technique
 Coverage criteria for GUI testing
 We also plan to explore the possibility of using
the event-based coverage criteria for software
other than GUIs.
 Object-oriented software
 Networking software
 The broader class of reactive software
Any questions?