This was the presentation @stueccles gave at the #firestarters event at Google UK. The event was about Agile Planning and Stuart talked about the origin of post-war production line processes, innovation for agencies, the lean agency and how to iterate successfully.

2. #firestarters
@stueccles

3. Holler Gram

4. We make new stuff
out of the internet

5. Selling Stuff
Making Stuff Useful

6. The Internet ate my business

7. Fragmented media:
more difficult/expensive
simply to buy attention

8. On-demand
Real time
Opt-in
A-la-carte world

9. Everything’s changed...
(Everything keeps on changing)
(It doesn’t show any signs of slowing down)

11. Digital possibilities seen through the
lens of traditional advertising...
All too often miss the real opportunities
thanks @garethk

12. Opportunities like...

13. Fiat eco:Drive

14. Garmin Connect

15. “Ideas that do”
Stop communicating
products & start making
communication products
Gareth Kay, Goodby Silverstein & Partners

16. Are we able to innovate
for our clients?

17. The Agency Innovation
Scorecard
D - Invention
Creation of a new idea or process
Noah Brier C - Innovation
Arranging the economics requirements
for implementing an invention
A + DiffusionAdoption and imitation

18. [images: twitter, youtube, flickr,
These guys are
napster, facebook, google, myspace,
really good at it
ebay, amazon, paypal, skype, slide,
spotify, apple, drop box]

19. In the latest turn of its wheel,
strategy becomes about how
to make existing institutions
as innovative as start-ups
Walter Kiechel III, The Lords of Strategy

20. Act like a start-up
MxM

21. • Rapid prototyping to test hypotheses
• Minimum Viable Product (MVP)
• Nail it then scale it
• Customer obsession (development)
• Iterative, metrics-driven & Agile
• Learn fast, don’t fail fast

22. Steve Blank
Startups are not a smaller
version of a large company

23. Large
Startup Company
The startup exists to search for a
scalable business model

24. Large
Startup Company
The large company exists to
execute a business model at scale

25. Thats great but we
have *clients*

26. There is no one-to-one
mapping
(But there is lots to learn)

27. ‘The Lean Agency’
Lean Agencies exist to search for new solutions to their
clients problems

28. The
characteristics
of a Lean
Agency

29. Iterative

30. Integrated

31. Evidence/
Metrics
Driven

32. Minimal
Waste

33. so about iteration..

34. Lets mass produce this SOB!!

36. Client Brief
Research
Strategic Development
Creative Brief
Creative Development
Production
Media!

37. Traditional forms of
planning and creative are
like betting on a hole-in-one

38. The holes in one are awesome

39. Kingsmill Bread Confessions
But most are not

40. Requirements
Functional Specification
Wireframes
Visual Design
Code
Test
Launch

42. Agile worked for us in
development but could
it work in creative,
design and strategy?

44. Skype Education Project
A new service to help more teachers use
Skype in the classroom

45. | Skype Education Project
Skype came to us
with a comms brief
Theyʼd noticed that teachers all over the world were
using Skype in extraordinary ways.
They wanted us to collect these stories and use them
to promote the service & inspire other teachers.

46. | Skype Education Project
We responded with some
ideas
And we proposed using an iterative, customer-
development focused approach.
This was an Agile software development project
driven by a continuous feedback loop of qualitative
and quantitative data.

47. Our initial
assumptions were
that it was all
about a lesson
planning tool
with examples,
ideas and tips on
integrating Skype
into learning

49. Use case #1: Invite an expert into the classroom
Stage 1 2 3 4
Touchpoint
How would you like to find an
How would you like to How would you like to involve How would you like to share
expert to take part in a
User Needs contact them and what would an expert in planning or the experience with the
‘Skype visit’? Some type of
you need to discuss? structuring the lesson? community?
marketplace?
Notes

50. “Getting out of the building”
George Mayo
Daniela Callegari Middle school language arts teacher in
Maryland, USA
Pioneer in technology and education
Enthusiastic about helping the project
Lisa Reid
Primary school teacher in Italy
No experience with Skype in the classroom yet
Desperate to connect and find other classes 6th, 7th & 8th grade science teacher in Ohio, USA
No experience with Skype in the classroom yet
Keen to start using Skype (has used EduSkypers)
Dan Sutch
Tracy Peterson
Technology teacher in Iowa, USA
FutureLab :: innovation in education Has introduced her classes to initial Skype sessions
In-depth knowledge of education & technology Keen to collaborate and extend usage of Skype
Open to ongoing involvement in project

51. We discovered immediately that their biggest obstacle
was actually finding other teachers who also used Skype

52. They also told us that teachers don’t have time to read lots
of lesson plans - video clips would be much more helpful

53. | Skype Education Project
We identified 3 key components
for an Alpha release candidate
Ongoing market research provided further evidence
that ʻFinding and sharingʼ was core. We pieced
together the simplest thing (MVP) that would allow us
to carry out lighter testing of our hypotheses at a
greater scale on the Internet.
We developed the first release in 2 iterations (4
weeks).

54. Insights gleaned from teachers’ websites

55. Skype educator
ecology
Sign
Mapping service Looking
up
Invite pioneers
propositions Willing
to mentor
for
Tags /
topics
Locatio
n Age
Expert Status group
wanting to Profile Questions &
help Name answers
Case D. Video/Comment
Conversatio studies
n
Suggestion
Video s
Filter
Belong Help
Skype
Search
educator
directory
Parents
IT
Finding
experts
Schedulin
g ‘How to’
Ask a guides School
API driven Link with Connect question policy
Time Tools other issues
B. Micro-Profile zones
teachers Sharing
Profile
search from
anywhere
Planning
Curriculu
m
‘How to’ guides
The Skype Teach
lesson Tools Resources
Experiences of existing
Feedbac community
k
A. Question & Answer
C. Search/Filter

56. We sketched up 4 pages of a directory service
which we tested with 7 teachers over Skype

58. Online surveys
- 55 teachers
- 7 countries
- 20+ subjects
Level of Skype teaching proficiency
Expert
30%
Intermediate
40%
Novice
29%

59. “What teachers want”
Find other teachers & share ideas 28%
Find a partner class 25%
Help other teachers 23%
How to use Skype 11.5%
Overcome obstacles to Skyping 9%
Practice Skype 3%

60. “Teachers who think concept
would be ‘very valuable’”
60% -
55% -
50% -
45% -
Search/Filter Video Q&A

61. | Skype Education Project
We were working within a
time and cost-boxed plan
We used various Agile techniques as a way of
managing delivery whilst retaining the flexibility to
respond to continuous feedback from testing.
We created a charter that defined key metrics for
success and a shared vision with Skypeʼs project
stakeholders.

62. Creating an MVP - high level beta stories
Overview People Resources Admin
I would like an admin
I want a directory overview I want to add a resource
I want to login to the site section for the Skype
manually educator site.
3 3 8 5
As an admin, I want to be
I want to create a profile I want to like a resource able to ban people who
misbehave
3 2 1
Tools
As an admin, I want to be
I want to add a Skype I want to view another I want to view a resource able to invite a teacher
Educator badge to my blog educators profile on it’s own page. with a unique url
2 3 5 5
As a member of the site, I I want to view the I want to be able to
I want to see all the moderate content, so that
want to invite other resources that another
educators in a directory bad stuff can be removed
teachers to join. educator has added / liked.
2 3 2 5
I want to filter the educator I want to share a resource I
directory found easily
8 1

63. A user journey through the service
My profile
A N Other profile
New ideas
People like me
I want to like a resource
The more people who mark an People like me
idea as good the higher the
2 ranking of the idea
Ideas I like
Ideas I might like
Good idea
I want to view the
resources that another Ideas I think are good are saved
educator has added / liked. into my profile so I can find them
again easily and other teachers
2 like me can find them
Mr Mayo
I can mark an idea as good on
the Skype teacher directory

65. Moving into production: refined wireframes

67. | Skype Education Project
With the first release, we start
getting analytics
The evidence we now start getting from both Skypeʼs
analytics package Omniture and Google analytics
provide a new stream of feedback and evidence.
The data points to 3 important site enhancements
currently being implemented, having also been
qualitatively validated with teachers in f2f interviews.

68. Analytics summary:
qualitative and quantitative working together
Beta hard metrics Beta soft metrics
‣ 1860 teachers signed up to the site ‣ Teachers like Skype in the classroom but it could meet their
needs better
‣ 60% of users from the USA
‣ Teachers want to use the site to connect with other teachers
‣ 2/3 of visits are new visitors to the site but find it difficult to find suitable matches with the current
‣ 17% return 2-3 times system
‣ 18% frequent return visits
‣ Teachers want more search tools and a wider range of topics
‣ High level of contact activity - 1,000 contact requests to describe themselves by
‣ 200 resources shared ‣ Teachers want greater granularity on location - probably in
part because of the high number of teachers in the USA
‣ 105 favourites
‣ Teachers find the resources section a useful source of
‣ 45 comments inspiration and information. This is often what they look at
first.

69. | Skype Education Project
What the analytics told us
• More than 1 in 2 users are connecting with another
• We saw only c200 resources shared. We researched this
further with teachers
• Teachers have started to ʻhack itʼ to create projects
• 60% of the traffic is from the US
• Return visits seemed low, but we realised that we were losing
the tracking when they connected via the Skype app

70. New objectives and revised
success metrics for phase 2
Phase 2 objectives
➡ Make connecting with another teacher easier and richer
‣ Teachers have more tools to find each other by
‣ Teachers can describe themselves in more detail
‣ Teachers can use alternative connection methods
‣ Teachers can find good contacts again
➡ Show teachers the best and most relevant resources
‣ Teachers see more relevant resources in their profile
‣ Popular resources are more visible
➡ Centre the site around projects rather than profiles
‣ Allow teachers to create and publish projects
‣ Surface relevant projects to teachers
➡ Increase teacher sign ups and number of visits
Phase 2 success metrics
‣ # new sign ups
‣ 1/2 teachers return to the site more than once
‣ Increase the number of teachers from outside the USA
‣ 500 connections per month - either email or Skype
‣ # of favourited resources increases
‣ # of bookmarked teachers
‣ # number of teachers who update their profile with additional information
‣ # number of teachers who embed the badge on their website

71. Data and evidence-based design
decisions for the new phase

72. lessons learned about
iterating..

73. Rinse
and....

74. It’s not iterative if you
only do it once....

75. Incremental

76. Iterative

77. Speed

79. If it takes too long
any iterative process
is indistinguishable
from waterfall

80. Software is so fast to
change it lends well to
iteration
(more some things are even faster)

81. Externalised + Rough

82. Sketch Prototypes

83. If you freeze an idea
If you freeze an idea
too quickly, you fall
too quickly, you fall
in love with it...
in love with it ...refine it too quickly
& you become
attached to it
The crudeness of early
models in particular is
very deliberate
#bdwny Jim Glymph, Gehry Technologies

84. Feedback

85. Get out of the building!

86. Learning
• The whole team learns
• What we learn is communicated
to clients
• What we learn is documented
• Metrics are important

87. Actionable metrics not vanity metrics
Pirate metrics... AARRR
RETENTION
ACQUISITION ACTIVATION REFERRAL REVENUE
Dave McClure

88. Please no! not more
fucking *testing*

89. A better idea/design may
exist but requires an
intuitive leap
Iterative optimisation
may only reach the peak of
a smaller idea
Local Maxima
props to @bokardo

90. Intuition Iteration

91. We always have a vision
that is clearly articulated,
big enough to matter, &
shared by the whole team
Our goal is always to
discover which aspects of
this vision are grounded in
reality, & to adapt those
aspects that are not
Eric Ries, Start-up Lessons Learned

92. This is all one grand
ongoing experiment
(But we can iterate our way to success)

93. Thank you
Stuart Eccles
@stueccles
stuart@madebymany.com
http://madebymany.com
Copyright Made By Many 2011