SlideShare uma empresa Scribd logo
1 de 27
Failure and Success in Software
Automation
Comparing 7 Projects

Oleksandr Reminnyi
Pre word
▪ The only identification of project being
successful is the fact that your customer is
happy.

PMbook
Understanding the goals
We want to save money and
time later on manual testing
We have a corporate
direction…
But the main is…
Automation is a BUZZ word
Average Automation Reality
Automation AND Skill Set
HOKKU
«Let`s automate everything!», the
client said.
And Mount Fuji trembled.
Customer skills
Should customer be mature in automation?
This is definitely not a prerequisite.
But it can influence…
ХОККУ

Читаю чужі тести і торкається моїх
скронь сивина…
То мозок проливається
Automation Dev. Team Skills
Does mature mean good automation here?
Not always.

Practices mean more.
Automation AND Time line
Automation and SDLC
When the automation might be successful?
Waterfall OR Agile?

Depends on automation layer;

Post release period!
Implementation OF one TC
Depends on complexity
Depends on number of steps
Depends on tested application

instrumentation

Average 4 to 20 man hours. Happens 50
Automation Toolset
Automation and Backend
Unit Tests

Really? Why?

Component/integration testing
Good job!
Service layer automation
Definite place to automate!
Automation And TOOLS
Can tool influence the success of
automation?
YES!
Automation of Mobility
Can mobile platform automation be buried before birth?

YES!

Silk Mobile $5000 per dev. per year,
20-30% failure rate only because of the tool issues

MonkeyTalk, Ranorex, etc.

Build in the agent, then find issues

Appium

maybe light in the end of the tunnel. But later

Consider out of device testing!
Documentation
Can automation exist without
regression test Cases suite?
YES!
At least they say it’s successful…
Different types of manual test cases
being Automated
-

Functional
Regression
Acceptance
…

Benefit is brought at
ones

once by Smoke
Automation Metrics
HOKKU
Sacura`s leaves are destined to fall down.
Once again we`ll see the Test Report in red.
Not only leaves are down.
What is usually Calculated?
ROI?
Nothing!
Sometimes:

Failure Rate
Suite Run Duration
Given architecture…

SUT

Automation Tool

UI

Services

…

Other SUT facilities that can be hooked but not
needed for automation tool access

Reporting
Base classes

Layer

Framework utilities

Test scripts
Test scripts

UI Maps – facilities to access UI (or other services) through
automation tools API

Layer

Etc.
Loading configuration

Test scripts Libraries
Test scripts
Functional

Logging

Layer

DB Helpers

Test scripts
Test scripts
Test scripts

Facilities to hook into SUT
through AF

Multi - layered architecture

Sometimes joined

Layer
Gave us possibility to collect More
metrics
Something obvious
Questions?
“I see the testing standards that do not tell
me how to create a test worthy of running."
DeMarco

Mais conteúdo relacionado

Mais procurados

Automation testing strategy, approach & planning
Automation testing  strategy, approach & planningAutomation testing  strategy, approach & planning
Automation testing strategy, approach & planning
SivaprasanthRentala1975
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POC
aakashmc
 

Mais procurados (20)

Selenium test automation framework design & development
Selenium test automation framework design & developmentSelenium test automation framework design & development
Selenium test automation framework design & development
 
Automation testing strategy, approach & planning
Automation testing  strategy, approach & planningAutomation testing  strategy, approach & planning
Automation testing strategy, approach & planning
 
Test automation with trends
Test automation with trendsTest automation with trends
Test automation with trends
 
Automation testing
Automation testingAutomation testing
Automation testing
 
Anupam_Resume
Anupam_ResumeAnupam_Resume
Anupam_Resume
 
Test Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & ToolsTest Automation Frameworks: Assumptions, Concepts & Tools
Test Automation Frameworks: Assumptions, Concepts & Tools
 
Automation_testing
Automation_testingAutomation_testing
Automation_testing
 
Automation
AutomationAutomation
Automation
 
Developing a test automation strategy by Brian Bayer
Developing a test automation strategy by Brian BayerDeveloping a test automation strategy by Brian Bayer
Developing a test automation strategy by Brian Bayer
 
Automation
Automation Automation
Automation
 
Test Automation failure analysis
Test Automation failure analysisTest Automation failure analysis
Test Automation failure analysis
 
Test Automation Made Easy
Test Automation Made EasyTest Automation Made Easy
Test Automation Made Easy
 
Automation test scripting guidelines
Automation test scripting guidelines Automation test scripting guidelines
Automation test scripting guidelines
 
How to select the right automated testing tool
How to select the right automated testing toolHow to select the right automated testing tool
How to select the right automated testing tool
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POC
 
Manual testers vs. automation testers – how QA organizations can bridge the gap
Manual testers vs. automation testers – how QA organizations can bridge the gapManual testers vs. automation testers – how QA organizations can bridge the gap
Manual testers vs. automation testers – how QA organizations can bridge the gap
 
Roman iovlev battle - JDI vs Selenide - Selenium Camp
Roman iovlev battle - JDI vs Selenide - Selenium CampRoman iovlev battle - JDI vs Selenide - Selenium Camp
Roman iovlev battle - JDI vs Selenide - Selenium Camp
 
Simple and Effective Development Process For Software v1
Simple and Effective Development Process For Software v1Simple and Effective Development Process For Software v1
Simple and Effective Development Process For Software v1
 
Katalon studio vs selenium comparision
Katalon studio vs selenium comparisionKatalon studio vs selenium comparision
Katalon studio vs selenium comparision
 
Test Automation
Test AutomationTest Automation
Test Automation
 

Semelhante a Failure and Success in Software Automation by Oleksandr Reminnyi

Semelhante a Failure and Success in Software Automation by Oleksandr Reminnyi (20)

Saksham Sarode - Innovation Through Introspection - EuroSTAR 2012
Saksham Sarode - Innovation Through Introspection - EuroSTAR 2012Saksham Sarode - Innovation Through Introspection - EuroSTAR 2012
Saksham Sarode - Innovation Through Introspection - EuroSTAR 2012
 
Susan windsor soft test 16th november 2005
Susan windsor soft test   16th november 2005Susan windsor soft test   16th november 2005
Susan windsor soft test 16th november 2005
 
Getting Started With QA Automation
Getting Started With QA AutomationGetting Started With QA Automation
Getting Started With QA Automation
 
Discover the power of QA automation testing
Discover the power of QA automation testingDiscover the power of QA automation testing
Discover the power of QA automation testing
 
Best practices for test automation
Best practices for test automationBest practices for test automation
Best practices for test automation
 
Test automation: Are Enterprises ready to bite the bullet?
Test automation: Are Enterprises ready to bite the bullet?Test automation: Are Enterprises ready to bite the bullet?
Test automation: Are Enterprises ready to bite the bullet?
 
It's Automation, Not Automagic
It's Automation, Not AutomagicIt's Automation, Not Automagic
It's Automation, Not Automagic
 
Testing in agile
Testing in agileTesting in agile
Testing in agile
 
Manual Tester To Automation Tester
Manual Tester To Automation Tester Manual Tester To Automation Tester
Manual Tester To Automation Tester
 
Basics of Scriptless Automation for Web and Mobile Apps (1).pdf
Basics of Scriptless Automation for Web and Mobile Apps (1).pdfBasics of Scriptless Automation for Web and Mobile Apps (1).pdf
Basics of Scriptless Automation for Web and Mobile Apps (1).pdf
 
Why test with flex unit
Why test with flex unitWhy test with flex unit
Why test with flex unit
 
Introduction
IntroductionIntroduction
Introduction
 
summary
summarysummary
summary
 
Reliable mobile test automation
Reliable mobile test automationReliable mobile test automation
Reliable mobile test automation
 
Sustainable Test Automation: Collaborate within Team
Sustainable Test Automation: Collaborate within TeamSustainable Test Automation: Collaborate within Team
Sustainable Test Automation: Collaborate within Team
 
Sustainable Test Automation: Collaborate within Team
Sustainable Test Automation: Collaborate within TeamSustainable Test Automation: Collaborate within Team
Sustainable Test Automation: Collaborate within Team
 
Robotic Process Automation with UiPath Studio
Robotic Process Automation with UiPath StudioRobotic Process Automation with UiPath Studio
Robotic Process Automation with UiPath Studio
 
A Modern Dive into QA Automation and Beyond
A Modern Dive into QA Automation and BeyondA Modern Dive into QA Automation and Beyond
A Modern Dive into QA Automation and Beyond
 
Appium Interview Questions and Answers | Edureka
Appium Interview Questions and Answers | EdurekaAppium Interview Questions and Answers | Edureka
Appium Interview Questions and Answers | Edureka
 
Relate UI automation & performance
Relate UI automation & performanceRelate UI automation & performance
Relate UI automation & performance
 

Mais de SoftServe

Mais de SoftServe (20)

Approaching Quality in Digital Era
Approaching Quality in Digital EraApproaching Quality in Digital Era
Approaching Quality in Digital Era
 
Digital Product Security
Digital Product SecurityDigital Product Security
Digital Product Security
 
Testing Tools and Tips
Testing Tools and TipsTesting Tools and Tips
Testing Tools and Tips
 
Android Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, ToolsAndroid Mobile Application Testing: Human Interface Guideline, Tools
Android Mobile Application Testing: Human Interface Guideline, Tools
 
Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...Android Mobile Application Testing: Specific Functional, Performance, Device ...
Android Mobile Application Testing: Specific Functional, Performance, Device ...
 
How to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps SolutionsHow to Reduce Time to Market Using Microsoft DevOps Solutions
How to Reduce Time to Market Using Microsoft DevOps Solutions
 
Containerization: The DevOps Revolution
Containerization: The DevOps Revolution Containerization: The DevOps Revolution
Containerization: The DevOps Revolution
 
Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist Essential Data Engineering for Data Scientist
Essential Data Engineering for Data Scientist
 
Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS Rapid Prototyping for Big Data with AWS
Rapid Prototyping for Big Data with AWS
 
Implementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should KnowImplementing Test Automation: What a Manager Should Know
Implementing Test Automation: What a Manager Should Know
 
Using AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and BeyondUsing AWS Lambda for Infrastructure Automation and Beyond
Using AWS Lambda for Infrastructure Automation and Beyond
 
Advanced Analytics and Data Science Expertise
Advanced Analytics and Data Science ExpertiseAdvanced Analytics and Data Science Expertise
Advanced Analytics and Data Science Expertise
 
Agile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric ApproachAgile Big Data Analytics Development: An Architecture-Centric Approach
Agile Big Data Analytics Development: An Architecture-Centric Approach
 
Big Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for InnovationBig Data as a Service: A Neo-Metropolis Model Approach for Innovation
Big Data as a Service: A Neo-Metropolis Model Approach for Innovation
 
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
Personalized Medicine in a Contemporary World by Eugene Borukhovich, SVP Heal...
 
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
Health 2.0 WinterTech: Will Artificial Intelligence change healthcare? by Eug...
 
Managing Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max MarkovManaging Requirements with Word and TFS by Max Markov
Managing Requirements with Word and TFS by Max Markov
 
How to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions SuccessfullyHow to Implement Hybrid Cloud Solutions Successfully
How to Implement Hybrid Cloud Solutions Successfully
 
Designing Big Data Systems Like a Pro
Designing Big Data Systems Like a ProDesigning Big Data Systems Like a Pro
Designing Big Data Systems Like a Pro
 
Product Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman PavlyukProduct Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
Product Management in Outsourcing by Roman Kolodchak and Roman Pavlyuk
 

Último

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
GenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdfGenAI Risks & Security Meetup 01052024.pdf
GenAI Risks & Security Meetup 01052024.pdf
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 

Failure and Success in Software Automation by Oleksandr Reminnyi