How to develop a successful Selenium Automation Program

1. How to develop a successful Selenium
Automation Program
Dave Rader - CTO
Sreekanth Singaraju – VP of Testing Services
DATE: 6/28/2012

2. About Alliance
Alliance Global Services is a software development firm that partners with
software, technology and information-intensive businesses on their mission-
critical work.
We plan, design, architect, build, test and implement software applications,
platforms and products that become a primary driver of innovation and revenue
growth for our clients’ businesses.
We are recognized for driving quality and speed-to-market when business
success depends on the software inside.
We operate agile development centers in the United States and
Hyderabad, India and have over 650 employees
We have partnered with more than 400 innovative clients in the
Information Services, Software and Technology, Insurance and
Healthcare industries
2

3. Agenda
• Definition of success
• Impediments in Achieving Best in Class
Automation
• Challenges & Solutions for Selenium Automation
• Key Takeaways
• Titanium xAFT Overview
3

4. Definition of Success for an Automation Program
Identifies defects of significance
Effective and adopted by Developers &
Testers
The framework should be of low
maintenance and should not take Maintain
significant effort to handle routinue able
changes to AUT
Automated test execution produces
Robust consistent and provable results.
Can be leveraged on different Re-
releases, configurations & Usable
environments.
Test Ability to run Automated test scripts
Independ
ence cases in any order.
Delivers measurable value in
terms of decreased cycles of
ROI
execution, decreased effort and
improved quality
4

5. Impediments to Achieving Best in Class Automation
Objectives Non-existent or Unrealistic automation objectives
Focused around technical goals rather than business goals
Automation Highest automation ROI areas not identified
Strategy
Subscale initiatives or insufficient commitment
Start-Up Tool licenses and framework creation costs
Costs
Up front Labor cost for implementing test cases
Expertise & Manual test teams lack development expertise
Assets
Knowledge of test case lifecycle mgmt. missing
Measuring Lack of a metrics program to measure alignment with overall objectives
Effectiveness
Evaluate effectiveness of testing programs and fine-tune strategy
5

6. Challenges with Selenium Automation
Object Ability to identify Objects and manage them to limit the expense
References and effort to maintain updates to UI
Coding Develop scripts in a consistent and maintainable format for
Standards easy maintenance and effective execution
Reporting Provide commercial quality reporting that enables testers to
rapidly identify risks inherent in software and provide
information to developers towards remediation
Framework Implement a framework and process that will streamline
management of common elements and resources in
automation development
Expertise & The ability to develop and maintain effectively automation and
Skillset related assets to deliver ROI on automation investments
6

7. UI Object Reference
Most of the Commercial automation tools have the provision to create and manage UI objects
in a centralized repository. Selenium has limited tools to address this leading to high-
maintenance scripts
• Object reference created using
XPATH, ID, Name, Class or CSS.
Common mistakes in constructing
Object references
• Improper selection of Object
selection property type
• Duplicate references to same objects
• Full length XPATH from root used for
object identification
• Leading to brittle references that
break when minor changes to UI are
implemented
• Significant maintenance effort when
UI is redesigned
//html/body/div/div/div[2]/ui/div/div[2]/div/h2/a
7

8. UI Object Reference – Recommendations
• Utilize common stored location
for objects using logical names
• Use Key-Value-Pair combination
for Logical name and Object
identification
• Logical hierarchies to group
objects for same features at the
same place
• Recommend UI developers use
frameworks that provide unique
identifiers for controls
• Use the smart identification
mechanism and use the
optimized XPATH
//div[@id=’icon-edit’/h2/a
8

9. Coding Practices
Test engineers that perform script development on Selenium may not have formal
development training leading to common coding errors. These lead to expensive maintenance
when the scripts need to be updated or unexplained behavior for execution
• Not using a design pattern or
development strategies to
selenium.type("name=j_username", develop framework and scripts
"user1");
selenium.type("name=j_password",
"user123); • No exception catching block in
selenium.click("divdiv[1]spanlogin the coding
");
Thread.sleep(30000);
• Use of Objects reference such as
selenium.click("class=Compose_Email");
path/xPATH in the code
• Use of hard coded values for the
data for inputs and validation
9

10. Coding Practices – Recommendations
• Using a design pattern and
development strategies like Page
Object Model OR Domain Driven
Design
try {
//Login to the system and Verify Email • Utilize code style/quality checking
selenium.type(userNameLocator,
userNamevalue); tools to address:
selenium.type(passwordLocator,passwordvalu • Exception handling
e);
selenium.click(loginLocator);
• Commenting
• Hard coding
selenium.isTextPresent(welcomeText);
} catch (Exception e) { • Use reusable coding practices for
Logger.Debug(("Element :["+ welcomeText+
"not found" + handling input data, validations,
"continuing text execution")); object references and common
selenium.click("Compose_Email"); library functions
}
• Utilize reference based object
synchronization for script to
synchronize with the app
10

11. Reporting & Metrics
Most of the Commercial automation tools provide extensive reporting functionality is not
available in Selenium and needs to be developed
• Poring over logs to identify
exceptions is time consuming
and error prone
• Execution metrics such as
coverage, script effectiveness etc
help fine-tune framework
• Test Execution meta data such
as environments, browsers,
pass/fail etc need to be extracted
from the basic logs
• Mapping of pass/fail data to
validation points needs to be
created
• Fatures such as screenshots are
completely missing
11

12. Reporting & Metrics
• Multiple levels of logging for
easy debugging and analysis
• Drill down Test Result
reports
• Automatic Emailing the Test
Result Reports
• Test Result Report
Integration with Test Case
Management Tools
• Screenshot capture on every
step or on exception
• Defect identification
effectiveness, designed
coverage and traceability to
features
12

13. Automation Framework
Poorly designed automation frameworks only transfer inefficiencies from scripts to the
framework while providing limited benefits
An example of a poorly designed
framework:
• Poor architecture design limiting
Automation flexibility of choosing target
Framework: Target browsers, target environments
Browsers, Test Data, and test data suites
Object References, • Lack of standards and guidelines
Reusable Functions, for reusable functions,
Exception Handling, components and object references
Reporting • Reporting features limited to logs
from Selenium
• Weak governance leading to
diluted framework effectiveness
13

14. Automation Framework
• Framework should be application-
independent.
Configuration • Easy to expand, maintain, and
Files perpetuate as the underlying
application changes
• Generic functionality like navigation,
API Support Test Data
link checking, login/logout and
reports should be developed as
Application reusable functions
Specific Business
Functions • Business functionality will be
addressed in test scripts/suites
Logging and Object • Test Data treated as a separate
Reporting Repository input into scripts
• Implement support for Windows File
Generic and Authentication dialogs
Functions
• Scripts should have minimal
external dependencies from:
– Browser versions
– Target environments
14

15. Expertise & Skillset
Currently there are a lot of good developers and good testers – but it is very hard to find good
automation Engineers that can develop effective automation
Developers • Facilitate the creation of
automation testers by
Automation creating a cross
Engineer functional/technical
Tester training
• Support them with
robust automation
frameworks and
reusable assets
• Ultimately Expensive to
hire, maintain and retain
resources
15

16. Key Take Aways
Existing Run scripts on more than one browser
Automation
Program Run Sonar and identify code quality issues
Metrics and reports from framework
Validate scripts can run multiple data conditions with no code changes
Change IDs for a few UI controls and evaluate impact on scripts
Governance and process on managing framework
New Review off-the-shelf automation frameworks first before building from
Automation scratch
Program
16

17. Titanium xAFT Overview
xAFT is a Selenium based Automation tool to test web based applications from an
Excel worksheet using English like language.
allianceglobalservices.com/innovation
Based on Open
and proven 30% productivity
technologies improvements over
Selenium
automation
Flexible &
Maintainable
framework Execute scripts
from Excel
Across browsers Titanium xAFT (eXcel Automated Functional Tester) is a
and web services robust enterprise automation framework that enables
technical and non-technical users to manage and leverage
automation suites
Web Mobile WebServices 17

18. Thank You
18