The document discusses testing best practices for rich client applications. It outlines the challenges of testing user interfaces and interactions. It then describes different levels of testing from ad hoc to crowdsourcing. Unit testing, continuous integration, and automated functional testing are explained. The current state of testing tools for Titanium is presented along with a demo. Future directions including more automation and crowdsourced testing are envisioned.

1. Testing Best Practices
Kevin Whinnery
Platform Evangelist
Appcelerator Inc.
@kevinwhinnery
kwhinnery@appcelerator.com

2. Testing Best Practices
• The Challenge of
Testing Rich Client
Applications
• Levels of Testing
• Titanium Testing Tools
• The Future of Testing
Titanium Apps
• Q&A

3. The Challenge of Rich
Client Testing

4. Rich Client Testing Challenges
A Few Challenges: Testing rich client applications
• Relevant tests? remains a challenge today due to
• Tooling? the visual and human aspects of
• Automation?
• Production client applications.
environment?
• Human
interaction?

5. How relevant are our tests?
• Model tier testing is highly
doable, and test friendly
• UI testing is hard to do in a
meaningful way:
• Does code work in a long-
running scenario?
• If I programmatically build a
UI structure, how close is that
to the real UI structure at
runtime?
• I can test object
properties, but reliably
ensuring visual integrity is

6. What tools do we use?
• In JS, client side test tools are
somewhat limited in functionality
and scope
• What is available is more geared
toward simple applications
• …and typically tied to the
browser, in the case of
JavaScript

7. How do we automate tests?
• Scripting user interaction is
difficult, especially in native
mobile apps
• Requires multi-platform
environment
• Requires some sort of
connection to a running
simulator or device, which must
also be scripted

8. How close are we to prod conditions?
• A simulator only goes so far
• Need to test on a variety of
devices
• Mobile is a hostile and volatile
environment:
• Data connectivity?
• Network speeds?
• Software/hardware
combinations (iOS has this
pain now too)
• Background processes?
• Geographic locations?

9. How is our “look and feel”?
• Does our app “look right” with
different data sets?
• Are our buttons responsive
enough, or do loading screens
take too long?
• Are there operations which are
unintuitive, or error prone?
• Are the properties of the UI we
are changing and animating
actually reflected in the UI how
we expect?

10. These problems can
all be overcome, but
testing can become a
classic struggle
between…

11. Time

12. Returns (Quality)

13. And Resources

14. Diminishing Returns
Consider: Test automation can become
• How long will I extremely expensive. Frameworks
support this need to be maintained, infrastructure
app?
• How useful is the needs to be created, and time needs
test automation I
am adding – how
to be spent writing, scripting, or
true to life are recording tests.
my test results?
• Automation
versus human Make sure the investment is
testing
justified.

15. Levels of Testing

16. Levels of Testing
Types of Testing: Depending on the scope, shelf
• Ad hoc life, and team size for every
• Automated Unit application, only certain levels of
Testing
• Continuous testing may be needed.
Integration
• Automated
Functional Testing You probably already know these by
• Crowdsourcing
some name, but let’s establish a
common vocabulary.

17. Levels of Testing
Types of Testing: Ad hoc testing by a single tester or
• Ad hoc group of testers may be sufficient for
• Automated Unit small applications with a short shelf
Testing
• Continuous life.
Integration
• Automated
Functional Testing
• Crowdsourcing

18. Levels of Testing
Types of Testing: Programmatic unit testing can
• Ad hoc provide a baseline comfort level with
• Automated Unit code in a project, giving the
Testing
• Continuous developer knowledge that public
Integration
• Automated
APIs have not regressed and are
Functional Testing working as expected.
• Crowdsourcing

19. Levels of Testing
Types of Testing: Continuous integration can make
• Ad hoc use of an existing suite of automated
• Automated Unit unit tests, and ensure that all test
Testing
• Continuous suites are passing as new code is
Integration
• Automated
committed to the project.
Functional Testing
• Crowdsourcing
Continuous integration becomes
especially useful as a team
increases in size and activity.

20. Levels of Testing
Types of Testing: Automated functional testing tests
• Ad hoc actual application interactions, and
• Automated Unit inspects the state of an application
Testing
• Continuous at known points.
Integration
• Automated
Functional Testing This type of testing is sometimes
• Crowdsourcing
referred to as “record and play”
testing.

21. Levels of Testing
Types of Testing: Crowdsourcing deploys an
• Ad hoc application to real humans around
• Automated Unit the world, who can test applications
Testing
• Continuous on different data
Integration
• Automated
networks, devices, and locations to
Functional Testing provide real-world feedback on
• Crowdsourcing
application functionality.
Can also be employed to provide
feedback on look and feel and other
human elements.

22. Where We Are Today
Types of Testing: Appcelerator (and our community)
• Ad hoc have developed test automation
• Automated Unit tools to provide a baseline level of
Testing
• Continuous comfort with application functionality.
Integration
• Automated
Functional Testing The testability of your application
• Crowdsourcing
can be greatly enhanced by
employing modular design.

23. Demo: Testable
Titanium apps with
behave.js

24. Where We’ll Be Tomorrow
Types of Testing: Appcelerator, the community, and
• Ad hoc our commercial partners are
• Automated Unit collaborating on testing solutions
Testing
• Continuous across the board.
Integration
• Automated
Functional Testing Visit with SOASTA and uTest in the
• Crowdsourcing
hallways!

25. To Sum Up
• Test automation is a
great tool for ensuring
quality
• For rich client
apps, useful tests have
always been a
challenge
• Choose the level of
testing appropriate for
the project
• Join us in reducing
barriers to testing and
building awesome tools

26. To Sum Up
Downloads/Resources:
• http://bit.ly/
• codestrong_src
• ti_test_slides

27. Kevin Whinnery
@kevinwhinnery
kwhinnery@appcelerator.com