These are the slides that Andrew MacLeod and Patrick Myles presented at the SPA conference in London, April 2009 http://www.spaconference.org/spa2009/sessions/session219.html

1. Testing Ajax Applications
SPA 2009
© 2009 CAPLIN SYSTEMS

2. Testing AJAX Applications
INTRODUCTIONS
2
© 2009 CAPLIN SYSTEMS

3. Who we are
 Patrick Myles
Head of Engineering, Caplin Systems
 Andrew MacLeod
JavaScript Architect, Caplin Systems
3
© 2009 CAPLIN SYSTEMS

4. How this talk came about
 We have a big Ajax application
 We use Agile
 We needed a way to test each release
This is the overview we wish
we’d had 2 years ago
4
© 2009 CAPLIN SYSTEMS

5. This is not a Selenium HOWTO
 Just enough Selenium to:
 understand the grammar
 promote a discussion on frameworks
 see how good Selenium is!
 Not enough Selenium to:
 lose non-programmers (e.g. QA engineers)
 get bogged down in Selenium nuts-and-bolts
 see how frustrating Selenium is!
5
© 2009 CAPLIN SYSTEMS

6. Frustration
Browsers were never designed as
platforms for automated tests...
...so browser test tools are a hack!
6
© 2009 CAPLIN SYSTEMS

7. We would like you to leave with...
 Confidence
 automated testing
 hands-on experience
 Ideas
 designing tests
 tool selection
 building a framework
 planning a test programme
7
© 2009 CAPLIN SYSTEMS

8. Testing AJAX Applications
THE PROBLEM
8
© 2009 CAPLIN SYSTEMS

9. The Problem
 How can we ensure that each release of
our application works everywhere?
9
© 2009 CAPLIN SYSTEMS

10. Manual Testing
DOM
Test Browser
Pack Application
Tester
(acting as the User)
 Repeat for each new feature
 Repeat for all supported platforms .
10
© 2009 CAPLIN SYSTEMS

11. Automated Testing
DOM
Test Browser
Script Application
Robot
(acting as the User)
 How intelligent is the robot?
 How human readable is the script?
 How much effort does the robot consume? .
11
© 2009 CAPLIN SYSTEMS

12. Tools
Test Behaviour Driven
FIT
Fitnesse
Framework Development
COM
Cucumber
WebTest
browser control Domain Specific
rspec Language (DSL)
Watir
Selenium hamcrest
TestNG
WebDriver
Mercury JSUnit
HttpUnit
WindowLicker
Windmill
Auto-Hotkey LoadRunner
DOM
browser control
Screen coordinates
12
© 2009 CAPLIN SYSTEMS

13. Workshop
Why are browser applications
so difficult to test?
13
© 2009 CAPLIN SYSTEMS

14. Testing AJAX Applications
CAPLIN: 5 MINUTE CASE STUDY
14
© 2009 CAPLIN SYSTEMS

15. Caplin Trader
 Quick demonstration

16. Caplin’s Timeline
2007 2008 2009
Scope
Smoke Test (continuous integration)
Functional Tests (nightly)
Performance Tests
Authors Apps
Reference Implementation
Bespoke Implementations (up to 3 in parallel)
Development
QA
Framework
Selenium Framework v1 Selenium Framework v2
16
© 2009 CAPLIN SYSTEMS

17. Our first test...

18. ... a more recent test
goal: Use product finder portlet
do Login (${username}=”user1quot;, ${password}=quot;testerquot;)
do CreateNewLayout
do Insert (${portlet}=“Product Finder”)
do AssertPortletPresent (${portlet}=“Product Finder”)
do Logout

19. Testing AJAX Applications
EXERCISE 1:
SELENIUM IDE: SEND AN EMAIL
19
© 2009 CAPLIN SYSTEMS

20. Objective
 To get familiar with the Selenium IDE
 To produce a simple test script
 To review the script afterwards
 Experience some Ajax testing pitfalls

21. Schema
interact
record
playback

22. User Story
“As a user,
I want to send an e-mail message
so that I can stay in touch with my friend”
22
© 2009 CAPLIN SYSTEMS

23. Golden Rule
Put yourself in the user’s position
 write the test first
 watch it fail
 fix it
23
© 2009 CAPLIN SYSTEMS

24. Tools: Firefox Add-ons
 Firefox Selenium IDE
 easy authoring tool
 Firebug debugger
 reverse engineering tool
 XPather
 locates UI elements
24
© 2009 CAPLIN SYSTEMS

25. Demo
 Open Firefox IDE
 Record
 Save
 Playback
25
© 2009 CAPLIN SYSTEMS

26. Setup…
 Disconnect from LAN
 Connect to CaplinSpa2009 wireless
network
 Plug USB key
 Open Firefox + Selenium IDE
 (set ?:Examplesuser-extensions.js)
 Navigate to http://spa2009.com/webmail
26
© 2009 CAPLIN SYSTEMS

27. Your Turn…
 Login:
 user: spa1
 password: caplin2009
 Send mail To: patrick@spa2009.com
 Insert appropriate actions:
 pause
 waitForElementPresent
 waitForEditable
 Save in /Exercise1-Basic/
27
© 2009 CAPLIN SYSTEMS

28. Selenium IDE Review
Pros Cons
 
easy to set up no explicit verification
 
easy to write locators are unintelligible
 
easy to run brittle
 
little Selenium knowledge unstructured
required  desktop orientated
 generates code!  feature must exist
28
© 2009 CAPLIN SYSTEMS

29. Testing AJAX Applications
SELENIUM: 5 MINUTE TUTORIAL
29
© 2009 CAPLIN SYSTEMS

30. Selenium Test Script
One step
per line...
action thing to optional
act on argument
30
© 2009 CAPLIN SYSTEMS

31. Actions
 Selenium
Selenium.prototype.doType(sInputLocator, sInputText)
Selenium.prototype.doClick(sLocator)
 Custom Framework
Selenium.prototype.doClickControl(sLocator)
31
© 2009 CAPLIN SYSTEMS

32. Locators
 Selenium
xpath
id
link
...
 Custom Framework
PageBot.prototype.locateElementByMenuBarItem(sLocatorText, oDocument)
32
© 2009 CAPLIN SYSTEMS

33. Accessors
 Selenium
waitFor
store ElementPresent
assert
verify
Selenium.prototype.isElementPresent(sLocator)
 Custom
waitFor
store
MessageCount
assert
verify
Selenium.prototype.getMessageCount(sLocator)
33
© 2009 CAPLIN SYSTEMS

34. Testing AJAX Applications
SCRIPTS:
INTERPRETERS AND RUNNERS
34
© 2009 CAPLIN SYSTEMS

35. Script: Machine vs. Human readable
Pianola
(acting as pianist)
Piano Roll
35
© 2009 CAPLIN SYSTEMS

36. After the break...
Linguistic Application
Constructs Abstractions
• locator
strategies
• structure
• application
• functions
operations
• control flow
• state
transitions
DOM
Source Browser
compiler Interpreter Runner
Code COM Application
36
© 2009 CAPLIN SYSTEMS

37. Testing AJAX Applications
COFFEE BREAK (10 MINUTES)
37
© 2009 CAPLIN SYSTEMS

38. Design Principles
Testing Ajax applications
38
© 2009 CAPLIN SYSTEMS

39. Design Principles
 Domain Hierarchies
 how to make my tests more readable?
 modeling vs. maintainability?
 how can I isolate application changes?
 Test structure and composition
 how do I avoid duplication?
 how do my tests map to my requirements?
 how can I make my framework scalable?
39
© 2009 CAPLIN SYSTEMS

40. Testing AJAX Applications
DOMAIN HIERARCHIES
40
© 2009 CAPLIN SYSTEMS

41. ?
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
41
© 2009 CAPLIN SYSTEMS

42. What to use for framework domain?
1. Don’t bother! ...
42
© 2009 CAPLIN SYSTEMS

43. ?
Test framework
Verification Domain Test framework
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
43
© 2009 CAPLIN SYSTEMS

44. What to use for framework domain?
1. Don’t bother...
Quick to get going
Can use off-the-shelf tools (e.g. Selenium
IDE)
Lots of duplication in tests
Maintenance nightmare
Brittle tests
Rapidly become hard to read and author
44
© 2009 CAPLIN SYSTEMS

45. What to use for framework domain?
2. Mirror of system domain...
45
© 2009 CAPLIN SYSTEMS

46. ??
System Domain (again)
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
46
© 2009 CAPLIN SYSTEMS

47. What to use for framework domain?
2. Mirror of system domain...
Usually in same language as System
Seems attractive – particularly if System is
well modelled on problem domain
Can “share” domain objects
Often ends up with either tight coupling...
...or duplicated code / abstractions on both
sides of the Browser/Verification divide
Using same language as System doesn’t
reinforce the domain separation
47
© 2009 CAPLIN SYSTEMS

48. What to use for framework domain?
3. Model or DSL that reinforces user’s
interactions and use-cases...
48
© 2009 CAPLIN SYSTEMS

49. ?
Business (User) Domain / DSL
(Inbox, compose, check mail, logout, etc.)
Test framework
Verification Domain
(Locate, input, verify, etc.)
Test tool
Browser Domain
(DOM, text nodes, clicks, etc.)
Application
System Domain
(OO language, may model business domain)
49
© 2009 CAPLIN SYSTEMS

50. What to use for framework domain?
3. Model or DSL that reinforces user’s
interactions and use-cases...
Easier to understand for non-programmers
Less brittle when underlying domains change
More natural abstractions
Requires more maintenance of different
abstractions
Can make it hard to implement complex
interactions
50
© 2009 CAPLIN SYSTEMS

51. Testing AJAX Applications
EXERCISE 2:
USING A FRAMEWORK
51
© 2009 CAPLIN SYSTEMS

52. Objectives
 Move test script closer to application domain
 Introduce logical names for screen controls
 Remove physical references to screen controls
 Show Selenium extensions
 Make tests more readable
52
© 2009 CAPLIN SYSTEMS

53. Selenium Framework
Application
Abstractions
• actions
• locators
DOM
Source Browser
Interpreter Runner
Code Application
COM
53
© 2009 CAPLIN SYSTEMS

54. Exercise 2 - Framework
 Demo
54
© 2009 CAPLIN SYSTEMS

55. Testing AJAX Applications
UCD TEST STRUCTURE
55
© 2009 CAPLIN SYSTEMS

56. User Centered Design (UCD)
 A powerful way to model and design a
system
 Like Agile, focuses on Users – their needs
and goals
 Typically used for UI/UX design, but
provides great models for test frameworks
 We use with a simple text DSL
56
© 2009 CAPLIN SYSTEMS

57. User Centered Design (UCD)
 Brings focus back to users and their
 Goals
 Tasks
• Actions
 Fits well with user story structure:
As a [type of user]
I want to [a task or sequence of tasks]
so that I can [achieve some goal]
57
© 2009 CAPLIN SYSTEMS

58. Use-Cases and UCD
UCD concepts map
well to use-case
levels
Goal
Task
Action
58
© 2009 CAPLIN SYSTEMS

59. Story Structure
Tasks
Goal Actions
Send Email Login Action
Action
Send Message Action
Action
Action
Logout Action
Action
59
© 2009 CAPLIN SYSTEMS

60. Testing AJAX Applications
EXERCISE 3:
RUNNING A UCD TEST
60
© 2009 CAPLIN SYSTEMS

61. Objectives
 Demonstrate UCD structure
 Show task re-use
 Implement a new task
61
© 2009 CAPLIN SYSTEMS

62. Old Structure
Test Action
Action
Action
Action
Action
Action
Action
Action
62
© 2009 CAPLIN SYSTEMS

63. UCD Structure
Goal Task Action
Action
Task Action
Action
Action
Task Action
Action
63
© 2009 CAPLIN SYSTEMS

64. Demonstration
 Goal/Task/Action hierarchy
 Task level convenient for
 Reuse
 Maintenance
 Communication
 Action level too small to be reusable
64
© 2009 CAPLIN SYSTEMS

65. UCD Structure
Send Email Login Action
Action
Send Message Action
Action
Action
Verify Sent Action
Action
Logout Action
Action
65
© 2009 CAPLIN SYSTEMS

66. A new story…
“As a user,
I want to receive an email
so that I can prove I have a friend!”
66
© 2009 CAPLIN SYSTEMS

67. Your Turn...
Receive Email Login Action
Action
Action
Verify Inbox
non-empty
Action
Logout Action
Action
67
© 2009 CAPLIN SYSTEMS

68. Testing AJAX Applications
SUMMARY
68
© 2009 CAPLIN SYSTEMS

69. Summary
 Selenium is not a perfect testing tool, but…
 Selenium extensions are effective
 actions
 locators
 accessors
 UCD is a good way to structure test cases
 goals
 tasks
 actions

70. Taking things forward…
 Develop a simple DSL for testers to use
 invoke tasks with parameters
 get rid of ugly HTML “Selenese”
 use a PATH mechanism to locate tasks
-- goal: use product finder portlet
do Login (${username}=”user1quot;, ${password}=quot;testerquot;)
do CreateNewLayout
do Insert (${portlet}=“Product Finder”)
do AssertPortletPresent (${portlet}=“Product Finder”)
do Logout