8. Four Simple Questions
● Why are we doing what we are doing?
● Who are we impacting?
● How are we making an impact?
● What are we going to do to make that impact?
18. ● Track device usage: No system for tracking of device status
● Track app usage: No system to know whether the “data collection
apps” are indeed used by the workers
● Instant Content Update: Content update on the device is very time-
consuming
● Remote App Management: Process to install / update apps is time-
consuming
Goals
20. Scale
What we’ll measure
Meter
How we’ll measure
Good Measurements
Benchmark
The current situation
Constraint
The investment
Target
The desired value
Credit: How to Establish Good Measurements, Competitive Engineering.
32. How?
● How should the goal change the actors’ behavior?
● How can actors’ help us in achieving the goal?
● How can actors’ prevent us from achieving the goal?
35. ● Diverge and Converge
Credit: Design Thinking.
Find Alternatives
36. ● What else could the actors do for us?
● Who else can help us and how?
● Who can obstruct us?
Questions
37.
38. Identify Key Priorities
● What are the possible obstructions which can stop us
before we start?
● Are there any high-value long-hanging impacts?
● What are the key assumptions to test?
41. Earn or Learn
● What is the simplest way to test this?
● Could we test it without software?
● Could we start earning with a partially manual process?
42.
43. Fitting the metrics
● Add the metrics as bullet points
● Rephrase the nodes to include the metrics
● Separate metrics table
● Add metrics as additional nodes
47. ● Impact = change in behavior
● Ask the right questions
● Measure the impact
● Make it visible
● Shared Document != Shared Understanding
Key Learnings
49. Leena S N
Head of Engineering @Multunus
@leenasn / leena.sn@multunus.com
50. ● Make an impact, not just ship software
● Impact Mapping book
● Impact Mapping Discussion Group
● Introduction to Impact Mapping
● Impact Mapping for an MDM product
● Design Sprint
● User Story Mapping
References
This talks is about how do we deliver software which creates impacts. Before we look at what is an impact, lets look at some stats
According to the Standish CHAOS report, only ~30% of the software projects succeed. Others end up either being in a “challenged” state or fail altogether because of various reasons such as over budget, over time or lack of alignment.
Standish Group primarily a IT research advisory organisation that focus on software project performance.
Water-Scrum-Fall is a very common pattern that we see. The HIPPO [the Highest Paid Person in the room] deciding the backlog and the rest of the team work towards the same only to realise later that what has been done so far is not in alignment with the market requirements.
Many teams try to solve this problem of “lack of alignment” with the project charter or vision document but visibility becomes a struggle as these documents exists in some repository, hardly known to anyone or updated by anyone.
If you really want to test, ask team members: “What is the goal of this project?”. You may be surprised to know how many actually know about it.
Impact Mapping is a "Strategic planning technique", explained in the book Impact Mapping by Gojko Adzic.
Software delivery can’t be done in a vacuum by the technical team, instead it is a collaborative effort between the end users, the business team and the technical team.
This is critical because the dynamic nature of software, adds to the complexity.
So what is required, is a “technique” which:
Focuses on collaboration among all the stakeholders
Visualises and communicates the assumptions clearly
Is fast and iterative so that changes can be adapted without much delay
.
Needless to say, its a collaborative activity, ideally done by the entire development along with stakeholders especially the decision makers.
To avoid the problem of wrong people in the meeting if team is too big, include those people who can make decisions:
Technical experts
Decision makers
We’ve been using Impact Mapping for quite sometime now and have seen good results with it so far.
In this talk I will be sharing my experience while working with one customer, The District Collector officer of Sabarkantha, Gujarat.
He approached us to develop an Mobile Device Management solution to control the Android devices which their health workers are using for collecting data.
These Android devices, which are distributed to the health workers, contain educational content [mainly Videos and Presentations] to create awareness among the villagers about:
Malnutrition
Child Mortality
Women Mortality
Vaccinations
The health workers collect the health data using “data collection apps”, which are installed on the device. The data later gets synced to the cloud for further analysis by medical officers within the Health Department.
Split the Impact Mapping into two sessions:
Prepare the map - where you identify the “why”
Create the map - where you fill in the rest
Its recommeded to do it two sessions to give enough respect for creating the base correct, i.e. “why” as the rest of the process is based on that
The reasoning our customer had was that as the number of devices increased, the officers started facing issues with the “managing devices in the field”, so approached us for providing an MDM solution.
Rather than having just conversations around the goals, you can use the feature list to arrive at the goals
The MDM space is so vast and has a set of predefined features for remote management of devices which includes these mentioned features.
You can consider this as a shopping list, as mentioned by Gojko.
By using the 5 Y’s technique, we can understand the reasoning behind the feature. The conversation that happens during these 5Y’s, helps people available in the room to get better perspective of the features.
This is a conversation that happened with the customer and you can see that at the end of this, we arrived at the real problem. With similar conversation we could identify multiple problems which helped us define the goals better
As mentioned above, these workers are “non-tech savvy” people. Even though training has been provided to them on how to use the app and why it is important, one of the problems the health departments found was that not enough data gets collected. The workers do complain about the lack of connectivity, devices being slow and then get hung, battery drains frequently etc. as the reason for not using the app or not syncing the data.
Once you identify the goals, the next step is to identify the measurements for the goals
Good measurements as the one which answers the below five questions:
We arrived at these measurements for each of the goal. You can see all the measurements are not there at this stage, thats ok. You can arrive at those at a later stage too.
We can’t solve all the problems at once. It is recommended to concentrate on one goal at a time rather than partially concentrating on many.
Give more votes or cash to the key business people in the room, to arrive at the right business goals.
As mentioned above, the priority is among the two categories of problems:
Tracking: Tracking device and app usage in the device
Remote Control: Managing the device remotely [eg: installing Apps, updating content remotely]
So we felt that it’s good to concentrate on Tracking problem first and learn the usage pattern before bringing in any kind of remote control. This data can help us to implement the right kind of remote management system.
For example, if the tracking data shows low usage for the apps, then we can start further conversations or conduct further training sessions for the workers. Instead if it’s a connectivity issue, then actions should be to improve the connectivity.
So we decided to concentrate on two goals, which would help us understand the device and app usage.
Instead of targeting all the devices [there are ~300 devices], we decided to do it on a smaller set and we came up with the first milestone i.e. Learn usage pattern of 100 devices
Once we identified the milestone, thats when we need to start with the map.
Gary Klein, a research psychologist famous in the field of naturalistic decision making, says that, to take better decisions during unforeseen events people should know the purpose of doing something. The research was done on emergency professionals which include firefighters, critical care nurses, pilots, nuclear power plant operators, battle planners, and chess masters to see how they make decisions in split seconds.
In software industry even though things are not like machine critical as above, but whats usual is unexpected events happening. If you have clear purpose there is a high probability that you will make better decisions during such unforseen events.
The preparation step explains why it's valuable instead of placing it around the features or scope
As the next step, we identified the “Who” of the impact map, which is the first level of the Map.
Who can produce the desired effect?
Health Workers - by using the device and the apps
Medical Officers - By providing the Health Workers appropriate training
Who can obstruct it?
Network Providers - through bad network
App Developers - Through bad UX
Who will be impacted with it?
The villagers - Better awareness for them about health
Use the Diverge and Converge of the design thinking to get as many ideas as possible.
Timebox it to 20 minutes
Ask the below questions to help the team focus on the actors and the impacts.
Once you’ve as many alternates identified, now it’s time for prioritising the same.
We used the voting mechanism for prioritisation as follows
In this step, start discussing the deliverables. Define the budget which can be either the maximum budget or length of the tasks. Come up with creative ways to validate the assumptions. Ask the following questions:
We need to be clear whether the goal is to increase revenue or is to learn about the market. It is recommended to keep the duration short if the goal is “learn”, to avoid investing a lot of money and effort. Ideal experiments can be for a few days to few weeks, and the Lean Startup way of “Build Measure Learn” is highly recommended for “learning” faster.
Try using Jeff Patton’s User story mapping to slice the deliverables in an iterative way.
During this discussion, we took majorly two decisions:
Supporting only the 4.4 version of Android initially so that we can start testing early
Deprioritized the reporting, initially decided to share it manually
This will be outside of the map. You can divide the white table into two sections, one for the map and another for the metrics
Mind Map for Impact Mapping
Session 1: Preparation
Step 1: Discover real goals
Step 2: Define good measurements
Step 3: Plan your first milestone.
Session 2: Mapping
Step 1: Draw the Map Skeleton
Step 2: Find Alternatives
Step 3: Identify Key Priorities
Step 4: Earn or Learn
Measure the Impact
In my experience this is the best way to introduce the lean concepts and the experimental mindset. I personally have found it hard to actually bring in the mindset, and faced a lot of resistance when it’s introduced quickly.
But with Impact mapping, what I found is that as the process is evolved through asking the right questions, usually the build-measure-learn ends up with the natural choice. Maybe its just my experience, but I’ve seen it working multiple times both for the engineering team and for the business people.
We had seen it working with:
Google Venture’s Design Sprint
Jeff Patton’s User Story Mapping [its at its early stage, but its good combination of tools]