1. Software Requirements Software Design Software Maintenance Software Development Software Testing Greg Spindler Principal Glen NoesenVice President Testing at the Speed of Business

2. About Today’s Presentation Greg Spindler Principal Glen NoesenVice President Enterprise Project Solutions, a division of SkillStorm, specializes in providing program management services process improvement and project management expertise that can drive initiatives across the full development lifecycle.

3. What the Experts Say Software Maintenance Software Testing “Software testing is often treated as accidental in industry, where it is sometimes barely done at all, often done poorly, and seldom done well. Testing is often cut when schedules and budgets overrun. “The need for software quality has been increasing dramatically, and new ideas like agile processes put a heavy emphasis on testing” Jeff Offutt - Professor of Software Engineering, George Mason University. "The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

4. What the Experts Say Software Maintenance Software Testing “We have as many testers as we have developers. And testers spend all their time testing, and developers spend half their time testing. We're more of a testing, a quality software organization than we're a software organization.” Bill Gates on trustworthy computing. Information Week, May 2002 "The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

5. What the Experts Say Software Maintenance Software Testing “Irreproducible bugs become highly reproducible right after delivery to the customer” Michael Stahl’s derivative of Murphy’s Law Jeff Offutt - Professor of Software Engineering, George Mason University. "The national annual cost of [poor] software testing is estimated to range from $22.2 to $59.5 billion." - US National Institute for Standards and Technology

7. Identify specific projects for improvement.

8. Organize to guide the projects.

9. Organize for diagnosis for discovery of causes.

10. Diagnose to find the causes. Provide remedies.

12. Identify the customers both external & internal.

13. Determine customer needs.

14. Develop features that meet customer needs.

16. Prove process capability

18. Establish measurement.

19. Establish standards of performance.

20. Measure actual performance.

21. Interpret (actual versus standard).

23. Identify specific projects for improvement.

24. Organize to guide the projects.

25. Organize for diagnosis for discovery of causes.

26. Diagnose to find the causes. Provide remedies.

28. Identify the customers both external & internal.

29. Determine customer needs.

30. Develop features that meet customer needs.

32. Prove process capability

34. Establish measurement.

35. Establish standards of performance.

36. Measure actual performance.

37. Interpret (actual versus standard).

39. Identify specific projects for improvement.

40. Organize to guide the projects.

41. Organize for diagnosis for discovery of causes.

42. Diagnose to find the causes. Provide remedies.

44. Identify the customers both external & internal.

45. Determine customer needs.

46. Develop features that meet customer needs.

48. Prove process capability

50. Establish measurement.

51. Establish standards of performance.

52. Measure actual performance.

53. Interpret (actual versus standard).

55. Identify specific projects for improvement.

56. Organize to guide the projects.

57. Organize for diagnosis for discovery of causes.

58. Diagnose to find the causes. Provide remedies.

60. Identify the customers both external & internal.

61. Determine customer needs.

62. Develop features that meet customer needs.

64. Prove process capability

66. Establish measurement.

67. Establish standards of performance.

68. Measure actual performance.

69. Interpret (actual versus standard).

71. Testing Foundation Software Maintenance Software Testing TMMi Foundation, Test Maturity Model Defect Prevention Test Process Optimized Quality Control Test Measurement Software Quality Evaluation. Advanced Peer Reviews Test Organization Test Training Program Life Cycle and Integration Non-Functional Testing Peer Reviews Test Policy and Strategy Test Planning Test Monitoring and Control Test Design and Execution Test Environment No documented QA processes 3 4 5 Managed/ Measured Initial (Ad-hoc) Managed Defined Optimization 1 2 Organized and Embedded Test Practices Measured and Aligned Test Practices Continuously Improving Test Practices Start-up Basic test Practices

72. Economics of Software Quality Software Maintenance Software Testing Costs Due To Lack Of Quality QA Budget as % of Development DECLINING QUALITY COSTS Testing Objective Maturity Cost and budget estimates are those of the presenter and are based on personal experience and research Many IT professionals think of testing as a stage that comes near the end of IT product development.   Testing should be done during almost every phase of the IT product development life cycle.

73. Software Requirements Software Design Software Development Software Testing Software Maintenance Attain a Competitive Advantage Increase Profitability Where Defects are Introduced Where Errors Are Found The National Institute of Standards and Technology (NIST) estimates about 70 percent of software defects are introduced in the requirements phase. The later defects are found, the more expensive they are to fix. Identifying the critical bugs earlier in the lifecycle reduces costs Source: NIST 2002 RTI Project 7007.011 Source: NIST 2002 RTI Project 7007.011 Source: NIST 2002 RTI Project 7007.011

74. Attain a Competitive Advantage Increase Profitability Where Defects are Introduced Identifying the critical bugs earlier in the lifecycle reduces costs Source: NIST 2002 RTI Project 7007.011 Source: NIST 2002 RTI Project 7007.011

75. Attain a Competitive Advantage Increase Profitability Where Defects are Introduced Where Errors Are Found Identifying the critical bugs earlier in the lifecycle reduces costs Source: NIST 2002 RTI Project 7007.011 Source: NIST 2002 RTI Project 7007.011 Source: NIST 2002 RTI Project 7007.011

76. Software Maintenance Software Testing Relative Cost to Repair Defects When Found at Different Stages of Software X is a normalized unit of cost and can be expressed terms of person-hours, dollars, etc. Source: NIST 2002 RTI Project 7007.011 RAISE (Reliability, Availability, Install Serviceability, and Ease of Use).

77. It’s Complex so we Focus Business Models SkillStorm Software Maintenance Software Testing Software Requirements License Model Knowledge Skills Talent Software Design Software Development Techniques Tools Fixed Price Projects Q.A. Process Software Testing Software Maintenance Build-Operate Transfer Model Testing Strategy Improved test coverage Hewlett Packard QTP Compatibility Methodology Solution Focus Areas IBM Rational Hybrid Model Testing Design Reduced execution time agile Planning Time & Material Microsoft VS2010 Waterfall Test Execution Decrease test Escapes Dedicated Test Center Model Micro Focus Control RAD Test Maintenance Improved test repeatability WatiN, Watir, Selenium Prototyping Improvement Incremental Optimized Measured Initial Managed Defined

78. Industry Challenge Software Maintenance Software Testing Core Beliefs "The surest foundation of a business concern is quality.” Source: Andrew Carnegie A Service Delivery Platform allowing you to focus not simply on formulating a strategy or a vision of the futurebut on building the distinctive organizational and operational capabilities to deliver uncommon results.

80. This years cost reductions become next years budget challenge

81. The cycle is constant

82. Testing productivity tools have been lagging behind the general market. QA organizations have an opportunity to implement positive change through innovation.

83. TIME BUDGET SKILLS Automation Challenges Software Maintenance Software Testing “The economics of the current automation tools don’t stack up. To make them work you need too many expensive technical staff. Without them you end up with poorly designed test suites that are not maintainable. Even with the correct level of technical input, the tools don’t yield scripts that are easily linked to the business function they are meant to test” Expensive technical staff Poorly Designed Not Maintainable No Linkage to Business functions    

86. Overcome budget restrictions and change your testing practices from risk management to risk mitigation! Time & Speed, Change Software Quality from a Cost Center to a Profit Center SPEED:Gains of up to 4X the productivity of your current testing resources.

87. Align your process for success Software Maintenance Software Testing Reduce Development & Maintenance Costs Attain a Competitive Advantage Improve Customer Satisfaction Reduce Cycle Time Increase Profitability Simplify and Optimize your Software Testing and Maintenance Deliver Solution Software Testing Software Maintenance New Project Software Development Software Requirements Software Design Improved Team Collaboration Reduced Cost of Implementation Scalable Proven ROI Easy to use Automate throughout the Lifecycle Future-Proof Strategy Process Tools Go from Serial to Parallel lifecycle and delivery

88. Keyword vs. Code Generation Software Maintenance Software Testing Code Generation Approach Traditional Framework Approach Requirements The time involved in training, process and test-script development is a major part of the automated testing investment. Keyword or Data Driven Frameworks on test tools like QTP, Rational, VS2010 and others, is Labor Intensive, Time Consuming and expensive. Axe completely eliminates the cost and technical sophistication required in developing and integrating with automation tools. An unrivalled platform for harnessing the power of automation, where testers can develop and automate highly sophisticated test cases without the requirement to acquire proprietary technical skills.

90. Improved Team Collaboration Software Maintenance Software Testing Automation Engineer Testing Expert Design test in Axe Excel Spreadsheet Architecture Design Generate Test Script with Documentation Object Mapping Run Test to ensure correctness & Business process Customization & Support Upload and commit to Test MGT Tool New Functionality A Clear Delineation Of The Responsibilities And Skill Sets Creating High Performance Teams Axe is a test automation and Acceleration platform solution, leveraging your current or planned technology investment in functional and automated testing.  Addresses the most common problems with test automation, Testers usually do not have automation expertise.  Axe clearly delineates the responsibilities and skill sets between Testers and Test Automation Engineers. 

91. Tester-centric Environment Software Maintenance Software Testing Tester-centric Environment Axe Excel Test Modeling Code and Documentation Generation Test Execution Axe Test Results Test Management “Truly Next Generation” A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools. 

92. Tester-centric Environment Software Maintenance Software Testing Tester-centric Environment Axe Uses Excel to define test scenarios, the capabilities of Excel combined with the tester’s knowledge to quickly create test scenarios, that can be easily replicated and changed to increase the coverage of test cases against the AUT. Axe leverages the functionality of Excels built-in features (such as date handling and manipulation) to create the code and generate documentation. “Truly Next Generation” A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools. 

93. Tester-centric Environment Software Maintenance Software Testing Tester-centric Environment Axe employs a small action set and simple spread sheet layout which can be easily mastered with one day of practical exercise based training. Only five core actions: Set, Get, Val, Load & Save. Axe, can be easily extended to create new actions to enhance the function of the generated scripts or overcome automation difficulties. “Truly Next Generation” A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools. 

94. Tester-centric Environment Software Maintenance Software Testing Tester-centric Environment Once test scenarios have been defined, Axe can generate documentation, object use statistics (coverage) and the coded automation scripts required. Supports leading test management and version control tools, Axe can create test items through the tool APIs and upload generated code. “Truly Next Generation” A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools. 

95. Tester-centric Environment Software Maintenance Software Testing Tester-centric Environment Axe generated scripts are executed against the AUT and controlled directly from Axe, the test management tool or a scheduler. Alongside execution tool logs, Axe produces a complementary “English” log showing actions on a step by step basis. Following the completed execution of the required test scenarios, Axe can upload results to the test management tool to complete the whole automation process. “Truly Next Generation” A holistic approach to automation which can integrate multiple vendors tools together to meet different business needs and leverage best in class tools. 

96. Code Sample Software Maintenance Software Testing (QTP/VBScript)

97. Documentation Software Maintenance Software Testing

98. Test Results Software Maintenance Software Testing

99. Test Model & Hierarchy Software Maintenance Software Testing Sub-test Login as user 1 Run Test001….Test002…Test003… ad infinitum! Test Case Login – enter user ID and Password Login – with no user ID Login – with no Password

100. Investment Leverage Software Maintenance Software Testing Integration and Compatibility

102. Ensures Repeatable Business Process Software Maintenance Software Testing Accurate Defect Replication: Speed: Regression: Platforms: Axe, clients and customers are able to decrease their time to market and deliver complete and fully tested business solutions to their customers and clients, with lower, predictable costs and unmatched speed.

103. Evaluation Criteria Software Maintenance Software Testing Reduce Development & Maintenance Costs Attain a Competitive Advantage Improve Customer Satisfaction Reduce Cycle Time Increase Profitability Ease of Use Reporting Capability RequirementsSelectionProof of Concept Customization Capability Platform Support Testing Requirements & Business process Functionality Vendor Qualifications Management goals 1. Skills of staff 2. Automation Goals 3. Test Requirements 4. Required Features

104. Client Example Software Maintenance Software Testing Client example 1200 test cases 4 release cycles/yr 4 testers 6 days to test Testing Delivery Delays Automation was the only answer Delivery Time Development Ongoing Releases As Release Cycles Mounted, Testing Became The Bottleneck

105. Clients Case Study Attain a Competitive Advantage Increase Profitability Developed 22,000 automated test cases in just under two years Axe Users since August of 2008 QA Team is made up of three Axe Testers (non technical) and one Axe Technician (expert on QTP as well as Axe) and one Developer (application support). No experience with automation were creating scripts and running automated tests within hours of their training. Software make up is 5 Axe licenses with 3 QTP licenses. Return on Investment was in-year as opposed to multiyear with QTP alone.

106. Axe Customers and Clients Software Maintenance Software Testing

107. Clients Testimonial Software Maintenance Software Testing Industry Leaders “Axe has clearly demonstrated its return on investment and proved extremely worthwhile for Serena Dimensions” “Within three months of adoption the Axe tool set has become a core part of our Agile development process allowing us to standardise our testing processes and reap the substantial cost and business benefits that Axe has proven to return” Mir Derakhshan – Vice President SCM R&D Serena

108. Clients Testimonial Software Maintenance Software Testing Industry Leaders “We have used Axe on a number of high profile engagements and in conjunction with a range of tools has helped us achieve a high-level of compliance and consistency in the delivery of our testing by re-use of tested components. Its design and architecture allows us to deliver Quality, flexibly and at speed with the minimum of maintenance and the requirement to write additional driver scripts” Andy Nichol, Microsoft Services (NHS and E-Gateway)

109. Clients Testimonial Software Maintenance Software Testing Industry Leaders I have used Axe for the past 3 years. This tool helped me introduce automation to a QA department that had tried going the automation route a few different times in the past but it never really took off. Axe gave me the ability to teach the QA team quickly. After only a couple hours of train the QA team was creating and executing USEFUL test cases in the current project. The one thing that we need to be kept in mind is my QA team was able to learn and apply Axe, while keeping up with their day to day tasks. The project calendar was not changed because we wanted to introduce automation. As far as I am concerned Axe has set the bar for all other automation tools. I have and I will continue to recommend Axe to all my peers. Erich Scheman – Fiserv

110. Clients Testimonial Software Maintenance Software Testing Industry Leaders “With the scope and complexity of recipient clients we need to deliver to, Quality is one of our most important criteria. Axe allows us to operate a continuous change-based environment. This ensures testing is no longer a constraint on our requirement to deliver the highest quality systems quickly to a large number of clients simultaneously” Alan Long, Microsoft Programme Manager

112. ProvenROI

113. Easy to use

114. Automate the Application Lifecycle

115. Future-Proof

116. Team Collaboration

117. Cost of implementation       Code generation is the “Next Generation” for automated testing  “Remember, testing is not what is ultimately important… performance is.”

119. ProvenROI

120. Easy to use

121. Automate the Application Lifecycle

122. Future-Proof

123. Team Collaboration

124. Cost of implementation Synchronization Maintenance Agility Lasting Value Code generation is the “Next Generation” for automated testing “Remember, testing is not what is ultimately important… performance is.”

125. SkillStorm Capabilities Attain a Competitive Advantage Increase Profitability Consulting Services Conduct assessments of Clients ’ current testing methods-processes-culture Recommendations for establishing or augmenting current process for SW testing Development of testing processes at the project, organization and business Level Prepare testing strategies and test plans for projects and client organizations. Provide testing resources for functional and automated testing. Provide business staff, testers, test teams and over-site for SW quality organizations. Test management for efficient testing, defect management, reporting metrics and tool selection. Complete outsourcing of test cases and test script development Complete test outsourcing         

126. Q&A Glen Noesen: gnoesen@skillstorm.com Greg Spindler: greg@bizexc.com

127. Free Offer For Webinar Attendees Axe Solution Demo On-line POC