1. @atlassian #atlascamp

2. Show Me the Numbers
Automated Browser Performance Testing
Chris Mountford
Senior Software Developer, JIRA
Atlassian
@chromosundrift
2

3. Chris Mountford
2005
3.4.1
185
16660
3

4. Chris Mountford
4

5. Chris Mountford
4

6. Chris Mountford
Inline Edit
4

7. Chris Mountford
Inline Edit
4

8. Chris Mountford
Inline Edit
Search
4

9. Chris Mountford
Inline Edit
Search
4

10. Chris Mountford
Inline Edit
Search
Detail View
4

11. Chris Mountford
Inline Edit
Search
Detail View
4

12. Browser Performance
Testing
Welcome!

13. Browser Performance Testing
“of specific interactions for a single user
Measuring the perceived performance
”
6

14. Browser Performance Testing
“of specific interactions for a single user
Measuring the perceived performance
perceived
”
7

15. Outline
• Intro
• Optimisation Techniques
• JavaScript Renaissance
Engineering
• Perception
• Performance Factors
• Gotchas
• Take Aways
• JIRA Perf Tests
8

16. Outline
• Intro
• Optimisation Techniques
• JavaScript Renaissance
Engineering
• Perception
• Performance Factors
• Gotchas
• Take Aways
• JIRA Perf Tests
8

17. JavaScript Timeline
A history
lesson

18. 1995
Classical JavaScript
10

19. Browser Wars
10

20. Browser Wars
Dark Ages
10

21. Dark Ages
Firebug
10

22. V8
Firebug
10

23. V8
Node.js
Firebug
10

24. Node.js
Hipsters
10

25. 11

26. 12

27. 13

28. “Why do we split everything
into so many files?”
13

29. “Why do we split everything
into so many files?”
“Performance.”
13

30. 14

31. “Why do we merge all the files
into large batches?”
14

32. “Why do we merge all the files
into large batches?”
“Performance.”
14

33. 15

34. “Why...”
15

35. “Why...”
“Performance!”
15

36. “Why...”
“Performance!”
15

37. “
Show me the numbers!
”
– Not Jacky Chan
Hey it’s
Jacky
16

38. “ If all we have are opinions, let’s go
with mine.
”
If we have data, let’s look at data.
Netscape
CEO
– Jim Barksdale
17

39. Factors That Influence
Browser Performance
sad panda

40. Mobile Web
• Everything is the same but worse
• network / radio power mode steps, sleep, etc.
• cpu / battery impact
• memory
• standards compliance
• tooling
19

41. Network
Time
20

42. Parse
Time
21

43. JavaScript Execution
• JS Engines
• native bind(), map() etc
• DOM Manipulation
• events
• JQuery
• Sizzle selectors
• micro-optimisation ☠
• Memory leaks
22

44. Blocking
23

45. Blocking
<script src=”wtf.js”>
23

46. Top Tools
Chrome DevTools
Firebug Lite
dynaTrace
24

47. Reflow

48. 26

49. inspired by @stoyanstefanov
27

50. Will It Reflow?
inspired by @stoyanstefanov
27

51. Will It Reflow?
inspired by @stoyanstefanov
27

52. Yes!
Will It Reflow?
inspired by @stoyanstefanov
27

53. But wait, there’s more!
• CSS Transforms
• Hardware acceleration?
• Data weight
• Concurrent request count
• Inline Base64 Encoded resources
28

54. But wait, there’s more!
• CSS Transforms
• Hardware acceleration?
• Data weight
• Concurrent request count
• Inline Base64 Encoded resources
28

55. Designing the JIRA
Browser Performance
Test Suite
Sweet!

56. JIRA Browser Perf Tests
30

57. JIRA Browser Perf Tests
Page
Objects
30

58. JIRA Browser Perf Tests
Page
Objects
Soke
Framework
30

59. JIRA Browser Perf Tests
Page
Objects
Soke
Framework
Bamboo
30

60. JIRA Browser Perf Tests
Page
Objects
Soke
Framework
Plugin
Bamboo
30

61. JIRA Browser Perf Tests
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

62. JIRA Browser Perf Tests
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

63. JIRA Browser Perf Tests
Koto
Charts
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

64. JIRA Browser Perf Tests
Koto
Charts
Psycho
Events
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

65. JIRA Browser Perf Tests
Koto
Charts
Psycho
Events
OnDemand
Analytics
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

66. JIRA Browser Perf Tests
Baselines
Koto
Charts
Psycho
Events
OnDemand
Analytics
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

67. JIRA Browser Perf Tests
Baselines
Koto
Charts
Vigilance
Psycho
Events
OnDemand
Analytics
Hacking
Page
Objects
Soke
Framework
Plugin
DB
Bamboo
30

68. Soke Interactions
• Define Timed Sections
• Number of Repetitions
• Personas
31

69. Soke Interactions
• Define Timed Sections
• Number of Repetitions
• Personas
Big ups to
Atlassian
PerfEng
Team
31

70. Soke Interactions in Scala
32

71. 33

72. 33

73. 34

74. Optimisation Techniques
• Batching
• Atlassian Plugins, No Worries
• Predictive Fetching
• Guided by Analytics
• Caching
• Watch for memory leaks
35

75. “
I don’t know... it just feels slow.
”
– a JIRA Product Manager
Show me the
numbers!
36

76. Perception is Reality
No, it only
seems that way

77. 38

78. Psycho Latency
• Ready for Action
• When does the User believe the app is waiting for her?
• Manual “psycho” event placement
• and maintenance!
• Yardstick tuning against a real system
• Optimisation Example: Execution Reordering
39

79. Gotchas
I’m gonna

80. click( )

81. Selenium / Webdriver
42

82. Selenium / Webdriver
• Chromedriver implemented by Chromium team
42

83. Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
42

84. Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft
42

85. Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft
• OK Browsers, follow the standard!
42

86. Selenium / Webdriver
• Chromedriver implemented by Chromium team
• mouse click on geometric centre of target
• W3C Webdriver wire protocol draft
• OK Browsers, follow the standard!
42

87. Selenium / Webdriver
43

88. Selenium / Webdriver
• Windows and IE process control
43

89. Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades
43

90. Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades
• Changing what you measure
43

91. Selenium / Webdriver
• Windows and IE process control
• Browser auto-upgrades
• Changing what you measure
• Prefetch cache benefit in JIRA 6.0
43

92. Take Aways
• Lots of factors
• Measure perceived reality
• Be holistic, quantitative, beware micro-optimisation trap
• Monitor and Maintain your performance suite
• Use Chrome DevTools etc.
44

93. Thank you!

94. For resources:
#atlascamp
@chromosundrift