Trung Nguyen presented on measuring agile team performance at DevDay Da Nang 2019. KMS Technology is a software development and testing company based in the US and Vietnam with over 1000 employees. Nguyen discussed collecting metrics from project tracking systems, source control management, continuous integration and deployment tools, and application performance monitoring to measure estimates, code quality, testing, and delivery. He provided examples of using metrics to identify technical debt, incorporate code quality, and measure the benefits of process changes. KMS Technology uses its own analytics tool called KA to combine and visualize metrics data.

1. DevDay Da Nang 2019
Measuring
Agile Team Performance
Trung Nguyen
Center of Excellent
KMS Technology

2. The U.S. based company
6 offices in the U.S. and Vietnam.
1000+ employees.
OUR EXPERTISE:
Software Development
Software Testing Solution
Startup incubator – Upstar Lab
Building world-class products for clients mainly from the U.S.
Providing superior independent software testing service.
Supporting product development projects in Vietnam:
About KMS
Established in 2009
QASymphony , Katalon, Kobiton, etc.

3. About me
KMS Center of Excellence member
Standard processes and best practices, advisories and
solutions, research and training for expanding technical
capability.
Marathon amateur runner
HCMC Techcombank 2018 - 21.1km finisher
HCMC Marathon 2019 - 42.2km finisher
Dalat Ultra Trail 2019 - 42.2km finisher
Danang Marathon 2019 - 42.2km (registered)
Vietnam Mountain Marathon 2019 - 70km (registered)
Contact
trungnguyen@kms-technology.com
Trung Nguyen
Director of Technology
KMS Technology Vietnam

4. DevDay Da Nang 2019
Why is Measurement Important?
Why Agile Teams struggle with measurement?
● Agile definitions of measurement are not straightforward
● Agile focuses on a product, not a project
● Data is all over the place without a unified view
● People are headed down the same road but having different expectations
“If you can’t measure it, you can’t improve it”
- Peter Drucker -

5. Objectives
1
2
3
Data Sources for Agile Metrics
What questions can metrics answer, and where do we get the data
to answer them?
Applying Agile Metrics
Case-studies that combining metrics to analyze and solve the problems
Agile Metric Tools at KMS Technology
What we build and use the Agile metrics at our company

6. DevDay Da Nang 2019
Data Sources for
Agile Metrics

7. DevDay Da Nang 2019
Software Development Lifecycle Tools
Project Tracking Systems (PTS)
Source Control Managements (SCM)
Continuous Integration & Deployment Tools (CI&DT)
Application Performance Monitoring Systems (APM)

8. DevDay Da Nang 2019
Project Tracking System

9. DevDay Da Nang 2019
Questions from Project Tracking System
WHO is working on WHAT?
WHEN THINGS get DONE in a GOOD way?
The members on
your team who get
assigned things
Tasks, bugs, or anything
that can get assigned
and tracked
Start/end
dates and
estimates
The same
as WHAT
End dates
affected by
the definition
of DONE
The team
was happy
with what is
took to get
there
● Estimates
● Task Volume
● Bugs
● Recidivism
● Tags
Key Data to Collect

10. DevDay Da Nang 2019
Source Control Management

11. DevDay Da Nang 2019
Questions from Source Control Mngt.
● Pull requests
● Denied Pull requests
● Merged Pull requests
● Commits
● Reviews
● Comments
● CLOC
Key Data to CollectWHO is working on WHAT?
HOW MUCH effort the work is taking?
The members on your
team who get assigned
things
Specific code changes to your
software products
WHO is helping WHOM?
How your team is collaborating
on their changes
How your team is collaborating
on their changes

12. DevDay Da Nang 2019
CI and Deployment Tools

13. DevDay Da Nang 2019
Questions from CI and Deployment Tools
● Test reports
● Total number of tests
● Percentage of
passed/failed tests
● Static code analysis
● Coverage percentage
● Code violations
Key Data to Collect
HOW disciplined the TEAM is?
HOW good your code is?
Build stats from CI show you how well
your team can follow the process
Also break down by team
member
HOW consistently you are delivering?
Maybe the most important
metric you can track
Through test results and static code analysis
can point to overall code quality

14. DevDay Da Nang 2019
Applying
Agile Metrics

15. DevDay Da Nang 2019
Story points are jumping all over the place
Sample-1: Identifying Tech Debt Trending
Adding volume to the pictureAdding bugsAdding tasks that move backward to the chartThe complete data over time

16. DevDay Da Nang 2019
Bugs aren’t trending downThe same data with variance decreases between bugs and other data
Sample-2: Incorporating Code Quality
Adding comments to the graph and finding an ugly trendEverything is trending in the right direction

17. DevDay Da Nang 2019
Sample-3: Measuring Benefits of Process Change
The graph the team ended with in the Sample-2. Everything is looking goodAdding velocity and releasesMost tasks estimated at 16 aren’t completedTasks are broken down into smaller bitsVelocity isn’t dipping anymore. The good/bad build trend is continuing to improveProductivity is consistent and releases are more frequent

18. DevDay Da Nang 2019
Agile Metric Tools
at KMS Technology

19. DevDay Da Nang 2019
KA - KMS Analytics

20. DevDay Da Nang 2019
Take-away
➔ If you can’t measure it, you can’t improve it
➔ Combine data from multiple tools
➔ Visualize data at the right levels
➔ Provide much data as possible to gain
insight your team
➔ Take ownership and measure team by
teamself

21. QUESTIONS
ANSWER
&
DevDay Da Nang 2019

22. DevDay Da Nang 2019
References

23. THANK
YOU!
DevDay Da Nang 2019