These are the slide from a webinar that we did together with BlackLine Systems on August 8 2012. During this webinar, Donna McCollum from Blackline Systems and Philip Lew from XBOSoft discussed lessons learned and best practices on how to set up test automation and performance testing for cloud software that is delivered with a SaaS model. Key words: Test Automation, Performance Testing, QA in the cloud, and QA for SaaS models. Features: • Learn best practices for QA, and performance, and automation testing for SaaS software • Hear about particular QA issues for managing financial SaaS software in the cloud • Understand the QA and testing strategies BlackLine Systems and XBOSoft implements to alleviate these issues For a recording of this webinar, please visit: http://blog.xbosoft.com/2012/08/09/performance-testing-and-test-automation-best-practices/ Key words: Test Automation, Performance Testing, QA in the cloud, and QA for SaaS models. Features: • Learn best practices for quality assurance and performance and automation testing for SaaS software • Hear about particular QA issues for managing financial SaaS software in the cloud • Understand the QA and testing strategies BlackLine implements to alleviate these issues For a recording of this webinar, please visit: http://blog.xbosoft.com/2012/08/09/performance-testing-and-test-automation-best-practices/

1. How to Achieve and Maintain High
Quality SaaS Software in the Cloud
Lessons learned and best practices on how to set up test
automation and performance testing

2. Agenda
• Fire side chat
– Introduction of the speakers
– Conversation between QA experts
– Q & A at the end

3. Howard Goldstein
VP of Marketing BlackLine Systems
• Built BlackLine’s marketing program
from the ground up.
• Oversaw BlackLine’s expansion
internationally into the EMEA and
APAC markets.
• Developed strategic partner
relationships with leading global
consulting firms.

4. Donna McCollum
QA Manager BlackLine Systems
• Managed BlackLine’s Quality
Assurance program for four years.
• Served as the key liaison to XBOSoft
overseeing functional and regression
testing.
• Previously was Manager of Financial
Technology at industry leader
SunGard Treasury Systems.
• Developed institutional trading
software.

5. Philip Lew
CEO and Founder XBOSoft
• Leads XBOSoft’s direction and
strategy as their CEO.
• Ph.D. research in software quality
and usability.
• Published in several IEEE and
ACM journal magazines.
• Presenter at various academic
and trade conferences.
• Industry leader in software
quality and measurement.

6. Housekeeping
• The audience will be on mute for a clear conversation
• Feel free to ask questions at anytime via the chat bar and we
will fit them in where appropriate.
• Contact information will be given at the end and you will
receive information on where to find the slide and the
recording.
Let’s get Started!

7. BlackLine Company overview
LA-based financial software provider, operating
globally
First to market with automated account
reconciliation software
475+ clients, many of which are in the Fortune 500
More than 58,000 users worldwide
‘BlackLine Certified Implementation Professionals’ from all
the major advisory/consulting firms and all around the world

8. GLOBAL
DEPLOYMENT
BlackLine OnDemand users
(89+ COUNTRIES)
BlackLine Offices

9. BLACKLINE CLIENTS (PARTIAL)

13. •SIMPLE, SECURE ONDEMAND/SAAS OFFERING
•REAL-TIME MONITORING AND REPORTING
•CONTROLLED PROCESS
•DATA INTEGRITY & INTEGRATION

14. BLACKLINE PROVIDES
EFFICIENCY, SPEED & VISIBILITY
REDUCED COSTS AND RISKS
CONTROLLED FINANCIAL CLOSE

15. ROI & CUTTING COSTS
• Auto-certification
MeadWestVaco Corporation
• Increased staff ROI: 495% Payback: 2 Months
productivity Cox Communications
• Reduced paper, ROI: 227% Payback: 6 Months
printing, storage Education Management Corporation
overhead, auditor fees ROI: 138% Payback: 10 Months
• Improved process
management Return on investment
is one of the most important
• Faster close indicators of value

16. Regression Testing
Image from: http://www.testfacebook.com

17. Regression Testing Key Points
• Impossible to test everything so implement
regression strategy based on context
• Use Risk Assessment based on actual data
• Exploratory Testing
• Defect Analysis

18. Performance

19. Performance Key Points
• Understand your users so you can develop
realistic performance scenario’s
• Benchmark and Improve
• Periodic Efforts

20. Test Automation
Image from simply-the-test.blogspot.com

21. Key Points
• What to automate depends on maturity of the
product and stability of modules and features
• Team is more important than the tool
• Set up a frame work

22. Best Practices & Lessons Learned
• Daily Build’s
• Goal 100% coverage, Risk Assessment when that’s not feasible.
• Fast and accurate: Accuracy is of primary importance, but
performance is a very close second.
• Automation is used where the product is mature and stable.
• QTP is used, but the quality of your people and framework of
the tests is more important.
• The remote XBOSoft QA team and the BlackLine development
and support teams work very closely together.
• The follow-the-sun workflow works very well.

23. Q&A

24. Contact info
Information on where to find the slides and the
recording will be mailed.
Please follow us on twitter @xbosoft
Request for info: services@xbosoft.com

25. THANK YOU!
www.XBOSoft.com
www.blackline.com