Introduction to automatic tests of Java GUI application (Swing/AWT)

1. GUI testing methods
Jemmy
Implementation
Features
Tools
Jemmy tutorial
Introduction to Jemmy testing framework
Pawel Prokop
pawel.prokop@adfinem.net
http://prokop.uek.krakow.pl
March 14, 2012
Pawel Prokop Jemmy Introduction

2. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Manualy testing
error prone
slow and not efficient
repeating test costs a lot
no place for TDD
boring for developers
(dinner syndrome)
flexible
look & feel fast feedback
Pawel Prokop Jemmy Introduction

3. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Recording user actions to exercise the application
efficient
fast preparation
fast execution
cheap
hard to modify during application development
common functionalities
no place for TDD
Pawel Prokop Jemmy Introduction

4. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Available tools for test recording
SWTBot is an open-source Java based functional testing
tool for testing SWT and Eclipse based applications.
http://www.eclipse.org/swtbot/
Marathon allows to record the script that can be modified
later manualy. Supports assertions.
www.marathontesting.com/
Jacareto is a tool that allows capture actions on
applications and replay them later.
http://jacareto.sourceforge.net/wiki
Pawel Prokop Jemmy Introduction

5. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Write code that simulates user actions
efficient
moderate slow preparation
parts of code may be reused
fast execution
easy to modify during application development
can be used with TDD
Pawel Prokop Jemmy Introduction

6. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Testing frameworks
UISpec4J is an open source framework to test Swing
applications.
http://www.uispec4j.org/
JFCUnit is an extension of JUnit that allows to execute unit
tests against Swing based interface.
http://jfcunit.sourceforge.net/
Jemmy is an open source framework that allows to
simulate user interactions with Swing applications.
https://svn.java.net/svn/jemmy~svn
Pawel Prokop Jemmy Introduction

7. GUI testing methods
Jemmy Recording tests
Implementation Testing frameworks
Features Summary
Tools
Summary
Manual testing Recording Framework
Efficiency - + +
Preparation time + + +/-
Execution time - + +
Price - + +
Reusability - - +
TDD - - +
Flexibility + - -
Pawel Prokop Jemmy Introduction

8. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Jemmy introduction
Jemmy documentation
not so many tutorials
not so many presentations and documents
good javadoc is enough to start
few samples
low Jemmy vitality
Pawel Prokop Jemmy Introduction

9. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Jemmy framework
How Jemmy works?
the same JVM as tested application
simulates user operations on the components by calling
events
events are stored on the QueueTool class and then provided to AWT
EventQueue
search components recursively by given criteria (caption,
name)
criteria defined as implementation of ComponentChooser
interface
Pawel Prokop Jemmy Introduction

10. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Jemmy framework
Jemmy in TDD approach
design UI interface and the components
implement jemmy tests to fit the interface
run failing tests
create implementation and re-run tests until they pass
Pawel Prokop Jemmy Introduction

11. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Jemmy Installation Eclipse
add jemmy.jar to the Java Build Path
Pawel Prokop Jemmy Introduction

12. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Jemmy Installation Netbeans
add Jar/Folder to Libraries
Pawel Prokop Jemmy Introduction

13. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Integration with JUnit
1 @Ignore
2 p u b l i c c l a s s CommonTest {
3 protected s t a t i c JMyApplication a p p l i c a t i o n = n u l l ;
4
5 p u b l i c CommonTest ( ) { }
6
7 @BeforeClass
8 p u b l i c s t a t i c v o i d setUpClass ( ) throws E x c e p ti o n { }
9
10 @AfterClass
11 p u b l i c s t a t i c v o i d tearDownClass ( ) throws E x c e p ti o n { }
12
13 @Before
14 p u b l i c v o i d setUp ( ) throws E x c e p ti o n {
15 a p p l i c a t i o n = new J M y A p p l i c a t i o n ( ) ;
16 a p p l i c a t i o n . startJMyForms ( ) ;
17 }
18
19 @After
20 p u b l i c v o i d tearDown ( ) throws E x c e p ti o n {
21 a p p l i c a t i o n . stopJMyForms ( ) ;
22 }
23 }
Pawel Prokop Jemmy Introduction

14. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Integration with Ant
Pawel Prokop Jemmy Introduction

15. GUI testing methods
Jemmy
Introduction
Implementation
Installation
Features
Tools
Integration with Ant
1 < p r o j e c t name= " JemmySample " >
2 < t a r g e t name= " j u n i t " >
3 < j u n i t printsummary= " yes " >
4 < c l a s s p a t h path= " l i b s / j u n i t −4.5. j a r " / >
5 < c l a s s p a t h path= " l i b s / jemmy . j a r " / >
6 < c l a s s p a t h path= " b u i l d / t e s t / c l a s s e s / " / >
7 < c l a s s p a t h path= " b u i l d / c l a s s e s / " / >
8 < b a t c h t e s t f o r k = " yes " >
9 < f i l e s e t d i r = " t e s t " i n c l u d e s = " ∗∗/∗ Test . j a v a " / >
10 </ batchtest>
11 </ junit>
12 </ target>
13 </ project>
1 a n t −f j u n i t . xml j u n i t
Pawel Prokop Jemmy Introduction

16. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Operators
Communication with UI controls is realized by operators
Jemmy defines own operators for every AWT/Swing UI
component
JFrameOperator
JDialogOperator
JButtonOperator
JCheckBoxOperator
JComboBoxOperator
JFileChooserOperator
...and many others
Operators can be extended by the developer
Pawel Prokop Jemmy Introduction

17. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Operators - small example
Yes/No dialog operator
1 p u b l i c c l a s s YesNoDialogOperator extends J D i a l o g O p e r a t o r {
2 p u b l i c YesNoDialogOperator ( ) {
3 super ( " C o n f i r m a t i o n " ) ;
4 }
5
6 p u b l i c v o i d pushYes ( ) {
7 new J B u t t o n O p e r a t o r ( t h i s , " Yes " ) . push ( ) ;
8 }
9
10 p u b l i c v o i d pushNo ( ) {
11 new J B u t t o n O p e r a t o r ( t h i s , "No" ) . push ( ) ;
12 }
13 }
Pawel Prokop Jemmy Introduction

18. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Operators - small example
Usage
1 // ...
2 @Test
3 public void test_some_functionality ( ) {
4 // ...
5 YesNoDialogOperator o p e r a t o r = new YesNoDialogOperator ( ) ;
6 assertNotNull ( operator ) ;
7 o p e r a t o r . pushYes ( ) ;
8 // ...
9 }
10 // ...
Pawel Prokop Jemmy Introduction

19. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Load application with jemmy
class reference
1 new ClassReference ( " jemmysample . JemmySampleApplication " ) . s t a r t A p p l i c a t i o n ( ) ;
direct call
1 JemmySampleApplication . main ( n u l l ) ;
Pawel Prokop Jemmy Introduction

20. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Concept depended approach
create classes that support some application’s
functionality:
1 c l a s s MyMainDialog extends J D i a l o g O p e r a t o r {
2 p u b l i c s t a t i c S t r i n g d i a l o g C a p t i o n = "My D i a l o g " ;
3
4 p u b l i c MyMainDialog ( ) { super ( d i a l o g C a p t i o n ) ; }
5
6 p u b l i c v o i d pushButtonCancel ( ) {
7 new J B u t t o n O p e r a t o r ( " Cancel " ) . push ( ) ;
8 }
9 }
and then use them from test suites:
1 @Test
2 public void test_dialog_can_cancel ( ) {
3 MyMainDialog d i a l o g O p e r a t o r = new MyMainDialog ( ) ;
4 d i a l o g O p e r a t o r . pushButtonCancel ( ) ;
5 }
Pawel Prokop Jemmy Introduction

21. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Application Shadow
1 public class JMyApplication {
2 p r i v a t e JFrameOperator mainFrame = n u l l ;
3
4 p u b l i c JFrameOperator getMainFrame ( ) {
5 r e t u r n mainFrame ;
6 }
7
8 p u b l i c i n t startJMyForms ( ) {
9 JemmySampleApplication . main ( n u l l ) ;
10 mainFrame = new JFrameOperator ( " JMyForms " ) ;
11 return 0;
12 }
13
14 p u b l i c i n t stopJMyForms ( ) {
15 JMenuBarOperator menuBarOp = new JMenuBarOperator ( mainFrame ) ;
16 JMenuOperator f i l e M e n u = new JMenuOperator ( menuBarOp . getMenu ( 0 ) ) ;
17 f i l e M e n u . pushMenu ( " F i l e | E x i t " ) ;
18 return 0;
19 }
20 }
Pawel Prokop Jemmy Introduction

22. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
The Component Choosers
How to find component?
using its index inside the container
textfield next to label
using Component Chooser implementation
NameComponentChooser uses a name property
PropChooser uses properties and methods to match a
component
Developer can define custom component choosers -
implementing ComponentChooser interface
Pawel Prokop Jemmy Introduction

23. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
RegexpFrameTitleChooser
1 p u b l i c c l a s s RegexpFrameTitleChooser implements ComponentChooser {
2 Pattern pattern = n u l l ;
3
4 p u b l i c RegexpFrameTitleChooser ( S t r i n g p a t t e r n ) {
5 t h i s . p a t t e r n = P a t t e r n . compile ( p a t t e r n ) ;
6 }
7
8 @Override
9 p u b l i c boolean checkComponent ( Component component ) {
10 i f ( component i n s t a n c e o f JFrame ) {
11 S t r i n g f r a m e T i t l e = ( ( JFrame ) component ) . g e t T i t l e ( ) ;
12 Matcher matcher = p a t t e r n . matcher ( f r a m e T i t l e ) ;
13 r e t u r n matcher . matches ( ) ;
14 }
15 return false ;
16 }
17
18 @Override
19 public String getDescription ( ) {
20 return pattern . pattern ( ) ;
21 }
22 }
Pawel Prokop Jemmy Introduction

24. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
ShowingComponentChooser
1 p u b l i c c l a s s ShowingNameComponentChooser extends NameComponentChooser {
2 p u b l i c ShowingNameComponentChooser ( S t r i n g name ) {
3 super ( name ) ;
4 }
5
6 @Override
7 p u b l i c boolean checkComponent ( Component component ) {
8 i f ( super . checkComponent ( component ) && component . isShowing ( ) ) {
9 return true ;
10 }
11 return false ;
12 }
13
14
15
16 }
Pawel Prokop Jemmy Introduction

25. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Timeouts
time values
(millisec-
onds) to wait
for
something to
be done by
the
application
Pawel Prokop Jemmy Introduction

26. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Timeouts
to delay simulated user behaviour
kept by org.netbeans.jemmy.Timeouts class
1 / / Set d e l a y b e f o r e key i s pressed d u r i n g t y p i n g
2 JemmyProperties . g e t C u r r e n t T i m e o u t s ( ) . setTimeout
3 ( " JTextComponentOperator . PushKeyTimeout " , 5 0 ) ;
each instance can have its own timeouts set
1 / / Set d e l a y between b u t t o n p r e s s i n g and r e l e a s i n g .
2 J B u t t o n O p e r a t o r b t n O p e r a t o r = new J B u t t o n O p e r a t o r ( " Process " ) ;
3 b t n O p e r a t o r . getTimeouts ( )
4 . setTimeout ( " A b s t r a c t B u t t o n O p e r a t o r . PushButtonTimeout " , 5 0 0 ) ;
Pawel Prokop Jemmy Introduction

27. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
Timeouts cont.
to wait until timeout exception is raised
1 / / w a i t up t o 5 seconds f o r J D i a l o g
2 JemmyProperties . g e t C u r r e n t T i m e o u t s ( ) . setTimeout
3 ( " D i a l o g W a i t e r . WaitDialogTimeout " , 5 0 0 0 ) ;
4 J D i a l o g O p e r a t o r albums = new J D i a l o g O p e r a t o r ( " D i a l o g Caption " ) ;
Pawel Prokop Jemmy Introduction

28. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
More timeouts
JComboBoxOperator.BeforeSelectingTimeout -
time to sleep after list opened and before item selected
JComboBoxOperator.WaitListTimeout - time to wait
list opened
ComponentOperator.WaitStateTimeout - time to
wait for item to be selected
JTextComponentOperator.PushKeyTimeout - time
between key pressing and releasing during text typing
JTextComponentOperator.BetweenKeysTimeout -
time to sleep between two chars typing
Pawel Prokop Jemmy Introduction

29. GUI testing methods Operators
Jemmy Concept depended approach
Implementation Component choosers
Features Timeouts
Tools Find components
find method vs operator constructor
static find method returns the operator or null
immediately:
1 @Test
2 public void test_some_functionality ( ) {
3 // ...
4 J D i a l o g O p e r a t o r myDialog = J D i a l o g O p e r a t o r . f i n d D i a l o g ( " E r r o r " , t r u e , t r u e ) ;
5 i f ( n u l l == myDialog ) {
6 / / no E r r o r d i a l o g i s c u r r e n t l y d i s p l a y e d
7 }
8 // ...
9 }
constructor blocks until component appears on the screen
or timeout.
1 public void test_some_functionality ( ) {
2 // ...
3 J D i a l o g O p e r a t o r myDialog = new J D i a l o g O p e r a t o r ( " E r r o r " ) ;
4 // ...
5 }
Pawel Prokop Jemmy Introduction

30. GUI testing methods
Jemmy
Screenshot
Implementation
Dump componenets
Features
Tools
Screenshot feature
Capture screen shot
1 // ...
2 @Test
3 public void testSomeFunctionality ( ) {
4 // ...
5 org . netbeans . jemmy . u t i l . PNGEncoder
6 . captureScreen ( " t e s t S o m e F u n c t i o n a l i t y . png " ) ;
7 // ...
8 }
9 // ...
Image modes
black and white
grayscale
colour
Pawel Prokop Jemmy Introduction

31. GUI testing methods
Jemmy
Screenshot
Implementation
Dump componenets
Features
Tools
Dump components in JVM
Dump components in JVM
Dump all components
1 // ...
2 @Test
3 public void testSomeFunctionality ( ) {
4 // ...
5 org . netbeans . jemmy . u t i l . Dumper . dumpAll ( " jemmy_example . xml " ) ;
6 // ...
7 }
8 // ...
Dump component
1 // ...
2 public void testSomeFunctionality ( ) {
3 // ...
4 Component comp = a p p l i c a t i o n . getMainFrame ( ) . getComponent ( 0 ) ;
5 org . netbeans . jemmy . u t i l . Dumper . dumpComponent ( comp ,
6 " jemmy_example_component . xml " ) ;
7 // ...
8 }
9 // ...
Pawel Prokop Jemmy Introduction

32. GUI testing methods
Jemmy
Implementation GUIBrowser tool
Features
Tools
GUIBrowser tool
shows all GUI objects in current JVM
shows components in hierarchy
allows to take snapshot in defined time delay
displays component members (name, caption, size...)
Pawel Prokop Jemmy Introduction

33. GUI testing methods
Jemmy
Implementation GUIBrowser tool
Features
Tools
Launch GUIBrowser tool
1 new ClassReference ( " org . netbeans . jemmy . e x p l o r e r . GUIBrowser " ) . s t a r t A p p l i c a t i o n ( ) ;
Pawel Prokop Jemmy Introduction

34. GUI testing methods
Jemmy
Implementation GUIBrowser tool
Features
Tools
Questions?
Questions?
Pawel Prokop Jemmy Introduction

35. GUI testing methods
Jemmy
Implementation GUIBrowser tool
Features
Tools
Links
1. Operators:
http://wiki.netbeans.org/Jemmy_Operators_Environment
2. Jemmy repository: https://svn.java.net/svn/jemmy~svn
3. Jemmy tutorial: http://wiki.netbeans.org/Jemmy_Tutorial
4. this presentation: http://prokop.ae.krakow.pl/projects/download/
jemmy_introduction.pdf
Pawel Prokop Jemmy Introduction

36. GUI testing methods
Jemmy
Implementation GUIBrowser tool
Features
Tools
Credits
1. flickr/ant/binux
2. flickr/clock/kobiz7
Pawel Prokop Jemmy Introduction