One of the biggest problems with code reviews is that they often derail developer productivity. Learn about the essentials of code reviews, where they are today, and where they can be using AI/ML technologies. With machine learning technology, code quality can be improved, and developers can focus on invention, rather than remediation.

2. © 2020 Perforce Software, Inc.
AUTOMATED CODE REVIEWS
USING AI AND ML
B R E N T S C H I E S T L

3. 3 | DevOps Next 2020 perforce.com
Confidentiality Statement
The information contained in this document is strictly confidential, privileged, and
only for the information of the intended recipient. The information contained in this
document may not be otherwise used, disclosed, copied, altered, or distributed
without the prior written consent of Perforce Software, Inc.

4. ABOUT ME:
Brent Schiestl
• Product Manager for the Version Control portfolio at Perforce Software
• 15+ years of experience in the Business Analysis / Product
Management space (formerly at Target Corp, Infor)
• Enjoy living at the intersection between customers and software
development teams
• Husband to an amazing wife, father of a “sassy” 6-yr old girl and a
“fearless” 3-yr old boy
• Hockey fan who resides in the “state of hockey” (Minnesota)
Pre-pandemic haircut!

6. 6 | DevOps Next 2020 perforce.com
Today’s Agenda
1
2
Introduction and Importance of Code Reviews in DevOps
Key Pillars for Valuable Code Reviews
3 How can AI/ML Enhance Automation and Efficiency of Code Reviews
Q&A5
4 Future of Automated Code Reviews

7. 7 | DevOps Next 2020 perforce.com
Introduction to Code Review
According to Wikipedia…
• Code review (sometimes referred to as peer review) is a software quality assurance activity in which one or several people
check a program mainly by viewing and reading parts of its source code, and they do so after implementation or as an
interruption of implementation. At least one of the persons must not be the code's author. The persons performing the
checking, excluding the author, are called "reviewers”.
Code Reviews Goals:
• Better code quality – improve internal code quality and maintainability (readability, uniformity, understandability, ...)
• Finding defects – improve quality regarding external aspects, especially correctness, but also find performance problems,
security vulnerabilities, injected malware, ...
• Learning / knowledge transfer – help in transferring knowledge about the codebase, solution approaches, expectations
regarding quality, etc.; both to the reviewers as well as to the author
• Increase sense of mutual responsibility – increase a sense of collective code ownership and solidarity
• Finding better solutions – generate ideas for new and better solutions and ideas that transcend the specific code at hand
• Complying with external guidelines – Code reviews are mandatory in some contexts, e.g., air traffic software

8. 8 | DevOps Next 2020 perforce.com
Standard Code Review Flow
Linting/static code
review is performed
A new software
build is created (CI)
Unit testing is
performed
Developer submits
pull/merge request
Code review is
created

9. 9 | DevOps Next 2020 perforce.com
Standard Code Review Flow
Linting/static code
review is performed
A new software
build is created (CI)
Unit testing is
performed
Developer submits
pull/merge request
Code review is
created

10. 10 | DevOps Next 2020 perforce.com
Benefits of Human-Based Code Reviews
Transparency
The entire team gets a better picture of what everyone is working on, their “style,” and even their level of productivity
Underscoring and Promoting Shared Values
Drives pride in work, making developers look forward to showing it to their colleagues
Becoming More Cohesive
Such reviews empower more experience developers to meet F2F with their peers, share best practices, praise, and improve
coding activities.
Building Self-Esteem for New Developers
Code reviews can be a great way for everyone to learn more about coding.

11. 11 | DevOps Next 2020 perforce.com
Code Review Checklist
Source: EvokeTechnologies

12. 12 | DevOps Next 2020 perforce.com
Code Review Best Practices
Know What to Look
for in a Code
Review
Build and Test –
Before Review
Don’t Review Code
for Longer than 60
Minutes
Check No More
than 400 Lines at a
Time
Give Feedback that
Helps (Not Hurts)
Communicate Goals
and Expectations
Include Everyone in
the Code Review
Process
Foster a Positive
Culture
Automate to Save
Time
9 Code Review Best Practices: Perforce.com

13. 13 | DevOps Next 2020 perforce.com
• Coping with scale in Agile/DevOps processes (Time/Money)
• Error Prone and subjective when done by humans (LGTM Syndrome)
• Only 13% of pull requests are rejected due to technical reasons (Limitations) 1
• Often leads to personal conflicts and sensitivity by the recipient of feedback
Human Code Reviews – Effective when done Effectively
1 McGill University, El Zanaty, et al.

14. 14 | DevOps Next 2020 perforce.com
Why is There a Need for AI/ML Automated Code Reviews?

15. 15 | DevOps Next 2020 perforce.com
Code Review Process using ML

16. 16 | DevOps Next 2020 perforce.com
Advancements in Code Reviews
“Find your biggest slowdowns by
pinpointing exactly where pull
requests get stuck on the journey
from open to deploy” – Code Climate
1. Automated Code Reviews
2. Code Coverage Analysis
3. Track progress against goals
4. Identify hot spots to focus

17. 17 | DevOps Next 2020 perforce.com
• Time to Open — The time between an engineer’s first commit and when they open a pull request in their version
control system. The Velocity data shows that this metric has the highest correlation with cycle time.
• Time to Review — The time between when a pull request is opened and when it receives its first review. Delays at this
stage incentivize multi-tasking, so Code Climate says you’ll want to minimize the time a merge of pull request is left
waiting for review.
• Time to Approve — The time between when a pull request receives its first review and when it is approved, also
known as the Code Review process. Clearly, as we discussed earlier, this is an area that needs analysis. You don’t want
LGTM reviews, but you also don’t want people avoiding reviews or spending too long on them.
• Time to Deploy — Any additional time following the merge or pull request approval, before the change reaches
production. This seems like it might be hard to measure in some environments, but it is absolutely worth doing.
Productivity Benefits of Automated Code Reviews with AI (Code Climate)

18. 18 | DevOps Next 2020 perforce.com
Advancements in Code Reviews

19. 19 | DevOps Next 2020 perforce.com
Advancements in Code Reviews
• Amazon CodeGuru - Find your most expensive lines of code and improve code quality
• Profiler helps developers find an application’s most expensive lines of code along with specific
visualizations and recommendations on how to improve code to save money
• Reviewer uses machine learning to identify critical issues and hard-to-find bugs during application
development to improve code quality
Open Pull Request
Add Amazon
CodeGuru
Reviewer as
reviewer
Amazon CodeGuru
Provides Intelligent
Suggestions
• Only supports Java (currently)
• Reviewer can be done on a per repository or per pull request basis
• All about willingness to pay!

20. 20 | DevOps Next 2020 perforce.com
• Lines are blurring between “traditional” static analysis and automated code reviews using AI/ML
• Rules and patterns vs. semantic meaning
• AI/ML in code reviews currently in the “early adopter” phase
• Existing tools are already available, integrated into the CI/CD landscape
• Be careful of the code review metrics you are tracking (unforeseen side effects)
• There will always be value in humans performing code reviews
Future of Automated Code Reviews

21. 21 | DevOps Next 2020 perforce.com
Classification of Advanced
AI & ML Testing Tools
COMING UP NEXT…
TRACK
Testing Tools
The Rise and Benefits of Robotic
Process Automation (RPA)
TRACK
Continuous Testing
Moving to Modern DevOps
with Fuzzing and ML
TRACK
DevOps & Code

22. © 2020 Perforce Software, Inc.
#devopsnext-devops-code
LIVE SLACK Q&A

23. Thank You!