Artificial intelligence (AI) has been changing the way software is tested and how humans interact with technology. AI predicts, prevents and automates the entire process of testing using algorithms. It will not only support and improve the models and test cases but also provide more sophisticated and refined form of text recognition and better code generators. Using AI will help to save time for testing and ensure a better quality software.
3. Agenda
3
What is Software Testing?
1
What is AI?
2
What challenges can AI solve?
3
Advantages and Disadvantages of AI
4
AI Tools for Software Testing
5
4. What is Software Testing?
International Software Testing Qualifications Board:
“Software testing is a process of executing a program or
application with intent of finding the software bugs.
It can also be stated as process of validating and verifying that a
software program or application or product meets the business
and technical requirements that guided its design and
development”
4
8. What is AI?
8
It is the science and engineering of making
intelligent machines, especially intelligent
computers to understand human intelligence,
but AI does not have to confine itself to
methods that are biologically observable
STANFORD
“Artificial intelligence (AI) is a self improving
enable horizontal layer that is solving problems
that were in the realm of science fiction for the
past several decades”
Jeff Bezos, Amazon.com Inc
Artificial intelligence (AI) makes it possible for
machines to learn from experience, adjust to
new inputs and perform human like tasks
SAS
Artificial intelligence is the boarder concept of
machines being able to carry out tasks in a way
that we would consider “smart”
FORBES
10. Applications of AI
10
Fraud Detection
• AI has the ability to identify
fraudulent behavior, as well as
identify what next pattern of
suspicious behavior will be
• AI algorithms can preempt such
fraudulent transactions and can
lead to huge costs savings for
banks and its customers
Meeting regulatory
requirements
• AI is used to ensure that
regulatory requirements are
met and that data is kept with
monitoring done on a real-
time basis. This allows issues
to be flagged a lot sooner
Boost customer
engagement
• AI will assist in the creation of
customized and intelligent
products and services, with
new features, more intuitive
interactions (e.g. speech) and
advisory skills (e.g. personal
financial management)
BANKING
Computer
aided
diagnosis
• AI is being used
extensively to read
and interpret
complex radiology,
pathology reports
to help doctors
arrive at early
High risk groups
identification
• AI is being fed huge volumes
of data historical medical
records that helps in
identifying whether a
patients is in a high risk
group for any particular
disease say stroke
cardiovascular diseases or
cancer
Epidemic out break prediction
• ML and AI technologies are also being applied
to monitoring and predicting epidemic
outbreaks around the world, based on data
collected from satellites, historical information
on the web, real- time social media updates,
and other sources
• Support vector machines and artificial neural
networks have been used, for example, to
predict malaria outbreaks, talking into account
data such as temperature, average monthly
rainfall, total number of positive cases, and
other data points
HEALTHCARE
10
12. Processes of software testing are similar to the processes used to
train AI
AI for Software Testing
AI applications use the output of the AI training process and
apply that to specific problems, such as recognizing a stop
sign and stopping a car, traffic lights … which consist of
inputs and comparing the outputs to expected results
12
13. AI Training data Process
Training AI systems is very similar to testing
AI training process assigned into categories: processing, sensing,
learning
13
14. What challenges can AI solve?
14
Ease of Authoring
and Executing
Tests
Releasing at the
Speed of
Development
Reducing
Maintenance and
Eliminating Flaky
Tests
Faster and More
Stable UI Tests
Continuous
Learning from
Production Data
Removing
Dependencies
15. Hundreds of attributes used to
identify elements
A few changes don’t break the test
Automatic get best location
strategy to successfully identify
15
Faster and More Stable UI Tests
16. AI’s self-healing mechanism can detect problems in the
failed tests before they even occur, fix tests instead of us
reacting to them
AI can figure out which tests are stable or flaky, analysis
what tests need to be modified to ensure test runs are
stable
Based on large numbers of test runs, AI can optimize the
wait times used in tests to wait for the pages to load and
also can handle tests running on different resolutions
Reducing Maintenance and Eliminating Flaky Tests
16
17. AI will start observing and learning how our customers are using the
product and can start creating tests based on real user data
AI will identify commonly used actions such as logging in/out of the
application and cluster them into reusable component
17
Continuous Learning from Production Data
LEARN BY OBSERVATION
(PRODUCTION)
AGGREGATE USER
ACTIONS INTO FLOWS
TEST PRODUCED
FROM FLOWS
18. Once we have authored some tests
and have run them consistently for a
period of time, the AI can start
recording all the server responses
When run the tests again, instead of
talking to a server or database, the test
will access the stored responses and
will continue to run without any
obstacles
18
Removing Dependencies
19. Author and execute tests can be done in a matter of hours
Use dynamic locators and the ability to easily create reusable
components
Integrate CI/CD systems easily with public and private grids
Nontechnical people can get involved in test
Increase collaboration within teams and encourages everyone to
own the test automation effort
Ease of Authoring and Executing Tests
19
20. With AI powering the transition to autonomous testing, reducing the
maintenance to a minimum, and creating more reliable tests, the ability
for teams to release faster is better than
Testers an maximize user coverage by connecting authoring of tests with
production apps mapping to real user flows
We have the ability to take a risk-based approach and base our
decisions on real data
We are now able to create more user scenarios in short period of time.
This means you can find bugs fast and release faster
20
Releasing at the Speed of Development
21. Improved accuracy and efficiency
Overcoming the limitations of manual testing
Benefits both testers and developers
Improving overall test coverage
Time & cost-saving
Advantages
21
Advantages and Disadvantages of AI
22. Advantages and Disadvantages of AI
Artificial intelligence software testers use the
concept of GIGO (Garbage in Garbage Out)
High costs
Can’t think outside the box
Disadvantages
22
24. Visual Testing
24
Baseline
<empty>
Image1
Image2
Image3
Result:
Image1
Image2
Image3
new
new
new
visualTest () {
simulate UI state 1
check (“check 1”)
simulate UI state 2
check (“check 2”)
simulate UI state 3
check (“check 3”)
}
1) Write app & test code 2) Run test 1st time 4) Baseline created3) Review results
Baseline:
Image1
Image2
Image3
visualTest () {
simulate UI state 1
check (“check 1”)
simulate UI state 2
check (“check 2”)
simulate UI state 3
check (“check 3”)
}
Image1F
Image2
Image3B
Result:
Image1
Image2
Image3
diff
==
diff
5) Update app & test code 6) Differences detected 8) Save baseline7) Review results
Baseline:
Image1F
Image2
Image3
Baseline:
Image1
Image2
Image3
3B:
Bug
1F – New feature
25. Segmentation algorithm, Convolutional Neural Networks and a
combination of algorithms
Can be directly incorporated into testing frameworks
Test results available in Test manager
25
Visual Testing
27. Regression Test
Focuses on reducing flakiness
Data driven testing
Automated test case generation
Reads production user access
27
28. Monitoring / Reporting
Different tools, frameworks and test for functional,
performance and security testing
Numerous test cases
Non prioritized test suite
Get meaningful data out of logs
28
29. One tool for code coverage for
different technologies
Prioritizes test cases
Functional, Performance, Security
Segregates test into smoke and regression
Failure prediction though log monitoring
Logs with reason for test failure
29
Monitoring / Reporting