Treating operational aspects of software as 'non-functional requirements' and 'an Ops problem' rather than a core part of the software product leads to poor live service and unexplained errors in Production. Deployability, recoverability, diagnosability, monitorability, and high quality logging are simply features of a software system, along with user-visible features surfaced via the UI, or a capability of an API endpoint. However, many Product Managers understandably feel uneasy about taking on the (necessary) responsibility for prioritising operational features alongside user-visible and API features. This session aims to bring Scrum Masters and Product Owners up to speed on operational features, empowering them to make effective prioritisation choices about all kinds of product features, whether user-visible or operational.

1. Demystifying operational features
for scrum masters and product owners
Agile Cambridge – 1st October 2015
#AgileCam
Rob Thatcher & Matthew Skelton

2. “Operational Features”
how to develop and test
prioritisation techniques

3. availability is the best feature

4. Operational Features

5. “the properties of a system
which make it work well in
Production”

6. Not PIMP MY RIDE
MORE
Greasy Mechanic

7. Not PIMP MY RIDE
MORE
Greasy Mechanic

8. Terminology

9. what happened to NFRs?
(non-functional requirements)

10. Non-
Functional
Functional

11. language impact

12. non-starter
non compos mentis
non-compete

13. nonsense !

15. holistic product view

16. How did we get to this?

17. admission: IT folk have been
guilty of making operational
features quite scary & mysterious

19. long lists of requirements
crazy test plans

20. poor explanation of needs
failure to engage stakeholders
gold-plating

22. de-mystify operational features

23. better approach
pragmatic and effective
rapid, safe, valuable

24. “the properties of a system
which make it work well in
Production”

25. Why value
Operational Features?

27. downtime:
$$$
reputation
($$)

29. non-linear increase in
complexity and problems

30. Internet of Things

31. we can no longer deal
manually with the scale/volume
of potential problems

33. agility and response to
incidents

35. remote car hacking:
security as an operational feature

36. HA + DR + Backup + Metrics + Diagnostics + …

38. think:
"when it fails, how will we recover?“
it will fail

39. How do we develop and test
Operational Features?

40. defined features
testable and measurable

42. ahead lie the ‘ilities’...

43. 1. What
2. How to test

44. Operational Hooks

46. Deployment Pipeline

48. Configurability

50. re-read config (SIGHUP)
text files in version control
inject settings – no ‘black boxes’

52. toggle features via config
“Postcode lookup unavailable”
 better UX

53. Deployability

55. immutable artefacts
concurrent releases (SxS)
symlinks

56. rapid
scriptable
simple failure modes

57. Maintainability

59. holding page as MVP!

60. live system component diagrams

61. modularity
ability to upgrade
version numbering (SemVer?)

62. BasketItemAdded
grep BasketItem

63. logging for insights

65. Testability

67. every component has a
/health endpoint

68. stubbed/mocked/faked
endpoints
test things individually

69. Recoverability

70. asynchronous service start
expect services to be erroring
logs are not wiped (rotated: okay)
avoid flooding logs

72. no nasty zombies after failures
MTTR more important than MTBF*
* for most kinds of F

73. Performance

75. run key 'hotspot' areas early
use a deployment pipeline
‘critical path’

77. early pipeline tests act as a
barometer for later
performance problems

79. derive transit time metrics

80. Monitorability

82. stream of metrics
transaction tracing

83. Resilience

85. Saboteur for network failure
testing
deployment pipeline

86. assume missing or failing
Chaos Monkey
don’t crash on HTTP 503

87. Scalability

89. concurrent workers
queues and bottlenecks
throttling is your friend

91. Security and ‘securability’

93. securability by practice
SSL certs & HEARTBLEED

94. Gauntlt
deployment pipeline

95. # nmap-simple.attack
Feature: simple nmap attack to check for open ports
Background:
Given "nmap" is installed
And the following profile:
| name | value |
| hostname | example.com |
Scenario: Check standard web ports
When I launch an "nmap" attack with:
"""
nmap -F <hostname>
"""
Then the output should match /80.tcps+open/
Then the output should not match:
"""
25/tcps+open
"""

96. Availability

98. “available but unusable"
synthetic transactions

99. special HTTP header: trigger
additional metrics/reporting

100. How the organisation affects
Operational Features

101. Budgets

102. bonuses:
story points delivered
tickets closed

103. Capex vs Opex
tax breaks

104. avoiding the Capex/Opex evil

105. Developers seen as more
valuable than Ops people
3x hiring bonus for Devs (!)

106. improved awareness in product
teams

108. share ownership and decision
making

109. features
end-user
operational
end-user

110. single product backlog

112. Product Owner on call for
incidents

113. tricky!
high degree of maturity
honesty about the product

114. Product Owner and Tech Lead
are both on the hook for
outages

115. AVOID
Product Owner for ‘user
features’ and Tech Lead for
‘operational features’

116. How to evaluate Operational
Features vs User Features

117. treat Ops team folk as another
user persona

119. alternatives to User Stories?

120. NOT:
"as a logging subsystem, I want..."

121. Metrics

122. Live: downtime, A/B for
operational aspects (speed)
Pre-live: time spent re-deploying

123. Metrics for better
conversations

124. metric-ify your delivery and test
infrastructure
 99.99% uptime, but 20
redeployments every time

126. Heuristics for operational features
30% of total product budget
30% of dev team time

127. holistic product view

128. MVP: ‘service unavailable’ page

129. test early for operational features
using a deployment pipeline

130. single product backlog:
(user) features +
(operational) features

131. availability is the best feature


132. Books!
operabilitybook.comoperationalfeatures.com

133. thank you
http://skeltonthatcher.com/
enquiries@skeltonthatcher.com
@SkeltonThatcher
+44 (0)20 8242 4103