This is served at Selenium Kansai

1. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Effective ROI
for getting started test automation
Woohyeok Kim
Rakuten, Inc.
@woosyume

2. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Woohyeok Aaron Kim
• Software Engineer
• Leading test automation team
• Scrum Master
• At Rakuten Ichiba Ranking Frontend
• Organizer
• Japan Selenium UC Kansai branch
• Korea Selenium UC (co-founder)
• Speaker
• SeleniumConf 2019
• Rakuten Technology Conference 2019
• Regional Scrum Gathering Tokyo 2020

3. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Why Test Automation
• The end of waterfall era
• New era satisfying with customer needs with agility is coming
• Quality Assurance is essential
Product
For customer

4. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Test Automation
• No one can deny the importance.
• Global companies already introduced test automation
• Introducing itself was done a few years ago
• They’re trying to improve their test application
• Performance improvement, Test Strategy etc.
• Japanese companies are getting started test automation

5. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
UI Test Automation
• UI Test Automation is so called pinnacle of all the tests

6. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Why we cannot start test automation
• Engineer
• Doesn’t have enough time (仕事が忙
しすぎる)
• Doesn’t want to perform it
manually (マニュアルテスト嫌や)
• Doesn’t know how to implement it
(なにで作ったら良いか分からん)
• Manager doesn’t allow it (したいけど、
マネージャーが。。)
• Cost issue
• Resource issue
• Manager
• Request cost so much
• Hope to concentrate on business
logic more (ビジネス案件に集中したい)
• Doesn’t understand what should
be done (何したらいいのか分からん)
• ROI is not reasonable (コスパが良く
ない)
• Believe we don’t need UT test

7. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Why we cannot start test automation
• Manager
• Request cost so much
• Hope to concentrate on business
logic more (ビジネス案件に集中したい)
• Doesn’t understand what should
be done (何したらいいのか分からん)
• ROI is not reasonable (コスパが良く
ない)
• Believe we don’t need UT test
Today’s topic is targeting
THE BIG MYTH

8. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
俺を説得してみろ!!

9. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Many Managers already know about merit of test automation
• Benefits
• Fast
• Programmable
• Reliable
• Etc.
• But they concentrate on only classic ROI

10. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Classic ROI
• Cost managers see
• Automation cost
• Dev + Maint + Execution + Result Analysis
• Manual Test cost
• Design + Execution + Maint
• ROI Formula they use
• (Manual Test cost – Automation cost) /
Automation cost
• E.g. -500% → NOT reasonable
They’re TOO familiar with classic ROI

11. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
What’s wrong?
• Test Automation cannot be calculated with classic ROI
• Automated Test can be repeatable
• Automated Test can be reusable
• Classic ROI only consider 1 on 1 comparison between AT and MT
• Actual ROI formula will be
(Manual Test cost – Automation cost) / Automation cost
222…

12. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Actual ROI
• Cover test cases automatically
• Increased productivity (生産性)
• Able to restore resource for manual test
• Increased test effectiveness (効率)
• Consistence, Stability
• Fostering team culture for Kaizen(文化作り)
• Resolve bottleneck
• Engineers more be happy
• Reducing Time to market
• Able to minimize release pace
Actual ROI show the effect beyond expectation

13. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
How to get approval (どうしたらいいか)

14. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Appeal current bottleneck on organization
• Physical bottleneck (物理的)
• It takes long time to perform test manually
• Time and resources are kept being wasted
• The resources used for the test make development plan be tight
• Psychological bottleneck (心理的)
• Depress engineer’s motivation

15. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Show your business is already in the red (すでに赤字ということを見せる)
• Deficit is arising per test execution (テストの時、赤字が発生している)
• Productivity is down (生産性が低くなっている)
• Wasted resources (無駄なリソースの消費がある)
• “We’re losing minimum XX hours. It’s over XXXXXX JPY when we monetize that”
• “We need to consider Re-execution cost for test”

16. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Show you can minimize automation cost
• Able to develop PARALLELLY (リリースが特に必要ない)
• Able to develop FAST because it has NO DEPENDENCY with other
projects (他プロジェクトの依存性がない)
• “I have FRIENDS having same trouble. Selenium User Community!”

17. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Killer technique (殺し文句)
“It’s probable that we cannot realize trouble even if particular part is
having trouble”
「今のままだと、トラブっていても、気づけない可能性があります」

18. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Then
• Finally, we can start test automation getting approvals
“Okay, please keep trying to improve our system”
「頑張ってください」

19. Japan Selenium User Community
日本Seleniumユーザーコミュニティ
Conclusion
• Value automated test has cannot be compared with it of manual test
• We cannot see proper return with classic ROI
• Managers are familiar with classic ROI
• Test Automation can provide us with many merits beyond expectation
• Productivity
• Kaizen culture
• Time to market
• Managers think the current status is reasonable
• But there is a possibility that it is NOT
• Propose optimized ROI for test automation