AI and ML solutions, whether commercial or open source, typically address unique use case or challenges. Learn about the categorization of testing tools with advanced AI/ML and get examples and existing tools for each of the use cases.

2. © 2020 Perforce Software, Inc.
Classification of AI and ML
Tools in DevOps
E R A N K I N S B R U N E R

3. 3 | DevOps Next 2020 perforce.com
Confidentiality Statement
The information contained in this document is strictly confidential, privileged, and
only for the information of the intended recipient. The information contained in this
document may not be otherwise used, disclosed, copied, altered, or distributed
without the prior written consent of Perforce Software, Inc.

4. ABOUT ME:
Eran Kinsbruner
• Chief Evangelist, Product Manager, and Author at Perfecto by Perforce
• Blogger, Inventor, and Speaker
• 20+ years in software development & testing
• Author of:
• The Digital Quality Handbook
• Continuous Testing for DevOps Professionals
• Accelerating Software Quality
• Twitter: @ek121268

5. 5 | DevOps Next 2020 perforce.com
Today’s Agenda
1
2
Introduction to AI/ML
DevOps Objectives and the Case for AI/ML
3 Classification of Key AI/ML Tools In DevOps
6 Q&A
5 Not Covered: RPA, Automated Code Reviews, Code Fuzzing
and ML, Defects Classifications, Test Management
4 Additional AI/ML Use Case Examples

6. 6 | DevOps Next 2020 perforce.com
AI and ML

7. 7 | DevOps Next 2020 perforce.com
• A compound of development (Dev) and operations (Ops), DevOps is the union of people, process, and technology to
continually provide value to customers.
• What does DevOps mean for teams? DevOps enables formerly siloed roles—development, IT operations, quality
engineering, and security—to coordinate and collaborate to produce better, more reliable products. By adopting a
DevOps culture along with DevOps practices and tools, teams gain the ability to better respond to customer needs,
increase confidence in the applications they build, and achieve business goals faster.
DevOps Objectives
Source: https://azure.microsoft.com/en-us/overview/what-is-devops/

8. 8 | DevOps Next 2020 perforce.com
The Impact of AI and ML on DevOps

9. 9 | DevOps Next 2020 perforce.com
What Roles Will AI and ML Play in DevOps?

10. 10 | DevOps Next 2020 perforce.com
DIFFERENTIAL VISUAL DECLARATIVE
Classifications of AI and ML Tools
Image-Based Learning
• Screen Comparisons
• Look and Feel
• UI Design, Accessibility
Comparing application
versions over builds:
• Recognizing changes
• Classifying the differences
• Bugs vs. features
ANALYTICSSELF-HEALING
Specifying Test Intent
• Goal-Based Automation
• Natural Language Processing
• Domain-Specific Languages
Autocorrecting Test Scripts
• Element Location
• Improved Maintenance
• Improved Robustness
Autocorrecting Test Scripts
• Root Cause Analysis
• Test Selection/Prioritization
• Test Flakiness

11. 11 | DevOps Next 2020 perforce.com
DIFFERENTIAL
• The tools in this segment aim to proactively and automatically identify code quality issues, regressions, and security
vulnerabilities through code scanning, unit test automated creations, etc.
Classification of AI and ML Tools in DevOps

12. 12 | DevOps Next 2020 perforce.com
VISUAL
Classification of AI and ML Tools in DevOps

13. 13 | DevOps Next 2020 perforce.com
DECLARATIVE
• Test Automation Methods:
• NLP (Functionize)
• MBTA (TestModeller.io)
• RPA (UIPath)
Classification of AI and ML Tools in DevOps

14. 14 | DevOps Next 2020 perforce.com
DECLARATIVE
Classification of AI and ML Tools in DevOps

15. 15 | DevOps Next 2020 perforce.com
SELF-HEALING
Classification of AI and ML Tools in DevOps

16. 16 | DevOps Next 2020 perforce.com
ANALYTICS
Classification of AI and ML Tools in DevOps

17. 17 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – Chatbots Testing

18. 18 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – Observability
Clustering Logs by RulesLogs Filtered By Time and Message

19. 19 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – AIOps
Source: ScienceLogic Source: Medium.com

20. 20 | DevOps Next 2020 perforce.com
Classification of AI and ML Tools in DevOps – TIA

21. 21 | DevOps Next 2020 perforce.com
DevOps is Still DevOps – AI/ML Should Fit in the Process
Classify
Classify the
pain and
category
Identify
Identify a POC
that makes
sense (pain
subset criteria)
Validate
Validate
solutions (skills,
org. fit)
Expand
Expand
solution to
solve the bigger
problem
Move
Move to
additional
pains

22. 22 | DevOps Next 2020 perforce.com
DevOps is Still DevOps – AI/ML Should Fit in the Process
Classify the pain
and category
Identify a POC
that makes sense
(pain subset
criteria)
Validate
solutions
(skills, org. fit)
Expand solution
to solve the
bigger problem
Move to
additional
pains
Pick a
solution
DIY – Do it Yourself
OSS – Open Source
COTS – Off the shelf

23. © 2020 Perforce Software, Inc.
UP NEXT…
Advancing the State of The Art in
AI and Testing

24. Thank You!