3. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
CREATING A CUSTOMER ENGAGEMENT METRIC DIY GUIDE
Jose Aleman, Director of Global Sales Operations at ACL Services.
FOREWORD
This paper is a practical guide for developing a single and tailored
organizational composite customer health metric.
The paper is targeted at managers and directors in the areas of operations
and customer satisfaction, although any employees evangelic about their
Customers’ experience will likely be interested. It is expected that B2B
Software businesses will find the most value in the paper, however, others
might also find useful tips.
The new concept captured within relates to the composite nature of a single,
tailored organizational metric, a metric that I have named the Customer
Engagement Metric (CEM).
Ideally, the paper will also foster discussion around Customer Success best
practices, so that other operations professionals can share their experience
and successes.
As Director of Global Sales Operations at ACL, one of my core
responsibilities is to provide insights into the levers that drive our business,
that of technology solutions for audit and risk. Having just pivoted into a
SaaS model, customer engagement is the fundamental leading indicator we
are tracking to drive and predict success. The methodology described in this
paper was refined while developing our own health metric: ICE (Index of
Customer Engagement). With over 15 years building expertise and leading in
corporate operations, I am passionate about sharing my learnings, and
hearing from other professionals on this topic too. More about my
background on LinkedIn: Jose Aleman.
Notes:
1. The required tools and methods used are intentionally low-tech to make
this new approach available to the broadest audience possible. Only MS
Excel and the ability to extract existing data from your internal sources as
CSV or Excel files are required in order to develop your organization’s CEM.
2. Your organization’s specific technology stack will likely support more
sophisticated data management methods. However, meaningful measures
are at the heart of the CEM; avoid measuring metrics simply because
they are easy to measure.
2
DIY GUIDE
4. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
THE WHY BEHIND CEM
The SaaS (Software as-a-service) business model has gained significant
momentum over the last few years. This paper contends that the key to SaaS
success is customer satisfaction, and this hinges on quality relationships.
Customers’ standards previously associated with the consumer world have
now crept into business. The new norm is for services to be available over
the internet anytime, anywhere, over any device. This trend, as well as lower
barriers to entry, and ease of service switch (no installations), make the
subscription business model a very attractive alternative for both software
vendors and customers.
Through subscriptions, software vendors enjoy a predictable revenue stream.
We know that this delivers premium valuation multiples and that this, in turn,
translates positively with the investment community. Further, from a customer
perspective, they enjoy the lower entry costs and the flexibility of adjusting
their spend as needs evolve.
While it is true that revenue becomes more predictable under a subscription
model, predictable is not the same as guaranteed. The only way a subscription-
based business can be successful in the long term is by retaining and growing
its customer base, and under a model where it is cheap and easy to switch
vendors, this is an ongoing challenge. Customer engagement is the key
differentiator. This means that SaaS companies are no longer purely in the
software business; they are in the relationship business as well.
The Relationship Business
Being in the relationship business means knowing about your customers,
understanding their drivers, listening to what they care about and
consistently delivering real value and outcomes. A first sale to a Customer is
not the culmination of the relationship but merely an increasingly fuzzy
milestone in a long term alliance.
Measuring customer engagement is a fundamental tool for managing
relationships with your Customers. Quality understanding around engagement
provides easy-to-communicate, actionable insights about your Customers. This
is why I am so excited to share my work around CEM.
More Resources
Books:
B4B: How Technology and Big Data Are Reinventing the Customer-Supplier
Relationship (J.B. Wood, Todd Hewlin, Thomas Lah)
Solve for the Customer (Dennis Pombriant)
The following companies offer Customer Engagement Metric solutions for
recurring revenue businesses: Zuora (www.Zuora.com), Totango (www.
totango.com), Gainsight (www.gainsight.com). You can find reviews for these
and others here: https://www.g2crowd.com/categories/customer-success/
products and a wealth of resources here: http://www.tsia.com/b4b
Search Terms: SaaS Valuation, Subscription Economy, “leaky bucket
customer retention”, SaaS Metrics, B4B
1
With the popularity of “Freemium” and other blended models.
2
As much as possible, it is a good idea to future proof your thinking. Try to avoid very short lived results even if they are important. You can always measure those separately from your CEM.
3
DIY GUIDE
5. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
YOUR CUSTOMER ENGAGEMENT METRIC
Why do you need one?
Measuring Customer Engagement is critical, but a Customer Engagement Metric is even more powerful. Here is why:
■■ It provides a common basis to compare Customer records.
■■ It allows for tracking changes over time using one common “currency”.
■■ It works both at a tactical level (looking into a specific Customer) and at a
strategic level (Organizational-wide “score keeping”)
■■ It moves the Customer satisfaction debate into action. A defined and
calculated CEM provides your organization with the rich tool needed to
move from contemplation into meaningful action around your company’s
culture and behavior toward customer satisfaction.
■■ Ultimately, through CEM, you can better move the dial on customer
satisfaction, retention, and income.
Can you just tell me what it should be?
Your Customer Engagement Metric needs to reflect the elements that are relevant to your business and to the type of relationship you have with your
customers, so it will be unique.
Building your CEM
Step 1: Define which metrics categories will go into your CEM
Your CEM will consolidate several metrics into one. I recommend that you work
with a number of key colleagues to define the sharpest measures that make up
your CEM. At ACL, this process included: Activation of the latest version of the
software, engagement with Tech Support, on-time renewals, and usage of our
latest features. (The CEM development process itself can be a powerful internal
animator for raising the importance of customer engagement within your
organizational culture).
Writing down a short list of the behaviors and results that you and your
CEM development team feel your organization should care about the most
is a good place to start. Consider what a truly engaged Customer would look
like and work back from there. For example, I suggest that an “engaged
customer” might:
1. Use our products / services consistently
2. Recommend us to others if given the opportunity
3. Have questions about how to maximize the value they get from our
service
4. Be up to date with their subscription renewal
5. Likely be interested in trying out new features
If the above is a good start for your engaged customer, your categories
would be: Product Usage (1), Customer Satisfaction (2), Use of Tech Support
or Forums (3), On-time renewal (4) and Upsell opportunities (5).
Once you have a list of categories, I suggest you stand back and verify that
each of the categories you have chosen actually measures a distinct aspect
of a Customer’s engagement. If you find there is significant overlap between
the categories, you will want to pick the best ones and shorten your list. Less
is more at this stage.
Step 2: Assigning relative weights to your categories
Depending on where your organization is in the SaaS lifecycle (Pivoting into
the model, partial SaaS offerings, 100% SaaS) you might find that some
elements are more important than others, so this is a good time to think
about what relative weight you will want to assign to each of these
categories.
A very straightforward way of defining and communicating relative weights
for different categories is to use the concept of percentage points. If you
think of your CEM as a numeric score between 1 and 100, you can give
each category a certain number of percentage points to divide the total 100.
As you go through this process, you might find it useful to actually draw a 10 by
10 grid and color-code the area that each category represents. This will make it
easier to visualize the relative weights.
For example:
Category 1
(40%)
Category 2
(30%)
10 points
“for playing”
Remaining 20
Graduated
Category 4
(10%)
Category 3
(20%)
Notes:
1. The fact that a given category has a certain weight in % points does not
mean that you cannot refine the score within that range, it only implies
that the maximum points you can get in any one category is bound to that
upper limit.
2. This upper bound per category is a useful feature, as you can ensure that
a Customer that is ranked highly has to be doing well in several categories,
as opposed to excelling in only one front.
4
DIY GUIDE
6. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
Step 3: Defining the specific metrics categories you will use
Once you have decided on your categories, you might consider unpacking
them further. For example, expanding on the first category, you could ask:
■■ What does consistent product use look like for you?
■■ Are there certain behaviors you care about more than others? For
example: Using newer features
■■ Are there metrics available anywhere in the organization that will
illustrate usage?
»» Can these metrics be linked to a specific account? User? Through a
user id? Email?
»» Are they time-stamped?
»» How frequently are they collected?
The goal is to find, for each category, the smallest number of metrics that
capture the essence of the category. Ideally, this would be one single metric.
As in the previous section, if there is overlap between what each metric
represents, I recommend you select the best one and leave the rest aside.
Here are some examples of metrics that might apply in your organization.
Category Example Metric Possible Sources
Product Usage Last login date
Product LogNumber of logins in the last month
% of Registered users that logged in in the last quarter.
Page views / sessions Google Analytics
Customer Satisfaction Last Net Promoter Score
Survey – Marketing system
Last C Sat Rating
Sentiment Analysis Social Media analytics
Referrals Marketing advocacy system
Tech Support Usage Number of cases in the last quarter
Tech Support
Ticket Tracking System
Distribution of cases in time
Number of escalations
% Cases closed within SLA
On time Renewal Customer Subscription Status
ERP / CRMLast transaction (1st purchase, renewal, upgrade)
Changes in ARR / MRR from last transaction
Upsell Opportunities Existence of open opportunities CRM
Now is the time to be realistic with your list. Which of these metrics are
available? Who collects them? Can you secure access? Etc. After careful
analysis, you might need to select alternative proxy metrics.
The following sections will give you some tips for manipulating your data into
metrics and your metrics into scores. However, there needs to be a minimum
set of data you can draw on before you can build a meaningful CEM.
3
Get ready to beg, borrow and steal to get what you need.
4
ARR = Annual recurring revenue; MRR = Monthly recurring revenue
5
DIY GUIDE
7. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
DATA MANIPULATION – THE OLD FASHIONED WAY
Regardless of the level of sophistication of your
organization’s BI stack, the information in this
section should be useful to build out a proof-of-
concept CEM with real customer data using only
data extracts and MS Excel. I recommend starting
with this approach even if more powerful tools
are available (Including ACL’s Analytics!). You can
always build in more sophistication when you are
ready to put your CEM into production. Starting
in excel will remove any potential technology
distractions. The objective is to build something
usable, immediately.
Step one is to find a way to directly relate the
component metrics that will form part of your
CEM to your customers. This can be a completely
straightforward thing to do (e.g. if your metric
data already lives on your CRM system and you
can simply extract it through a standard report) or
it might require a few steps.
Below are the basic principles; you might prefer
to progress immediately to the example.
1. Identify the source system where the data you
want is stored. Find out who manages that
system and what kind of extract capabilities it
supports.
2. Understand how the data is connected to the
Customer “object”.
a. Is the data linked to a user that can be
associated with a Customer via an ID or an
email address / domain?
b. Is the data already associated with an
“Account” through a system ID?
c. Is that ID stored as a Key anywhere else
–ideally, your CRM?
3. Extract the source system data. I suggest you
save your extract template and document it so
that the process is repeatable. Make sure to
get whatever IDs are available, they will come
in handy later.
4. Group your extracted data around your ID and
(likely) a date dimension.
5. Associate your grouped data with your
Customer ID.
Data Manipulation Example:
Tech Support Tickets
Suppose one of your component metrics is:
number of Tech Support tickets over the last X
months. Also, imagine that your ticketing system
is not integrated to your CRM. Your Ticketing
System is most likely able to produce an extract
that looks like this:
Ticket Table:
Your case ID is valid only within your Ticketing
System, and there is no direct link to Accounts,
however, you do have email addresses, so you
could establish a link between Cases and
Customers using those, as follows:
Your CRM system will likely have an Account
table and a Contact Table that contain the
following:
Account Table:
Contact Table:
You determine that you cannot get from your
Ticketing System directly to your Account, but
you can go through your contact table to get to
the Customer.
Ticketing System CRM
Case
Case ID
Email
Contact ID
Account ID
Email
Account ID
Account Name
Contact Account
To actually make the connections, you will want
to get familiar with 2 really useful excel formulas:
VLOOKUP and GETPIVOTDATA. There are a lot
of great resources online for this, both from
Microsoft and for 3rd parties so if you need a
refresher go ahead and google those terms.
To do this in excel, follow these steps:
1. Use VLOOKUP to “append” Account ID to
your Source Data. Use Email as your search key
and always use the parameter “False” to make
sure you get exact matches only)
Ticket Table with Appended ID
Create a Pivot Table so that you count unique
cases by Account by Month. The result should
look like this:
Pivot Table
2. With the data in this format, you can use
GETPIVOTDATA on Account ID to get the
cases for any given month, or for the period
total.
This technique can be used with all kinds of data
sources to uncover “hidden” links between data
sources.
Notes:
1. When using Salesforce.com IDs, always use
the 18 Character version of the ID, as the
standard ID is not interpreted as unique by the
excel functions used.
2. The technique described here can also be used
in more complex scenarios, where there is no
field that can be used as a key that can be used
to link systems by using excel formulas to derive
a key, for example, by stripping all the characters
to the right of an “@” character on an email
address to generate a “domain” with the
formula: =RIGHT(A1,LEN(A1)-FIND(“@”,A1,1))
Where A1 holds the email address.
6
DIY GUIDE
8. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
FORMULA HACKS FOR CREATING YOUR CEM
Base 100
The basic hack for making your life easy when creating a composite score is
to choose a base that is easily understood and calculated. I recommend
sticking to a 0 to 100 metric as this will give you enough granularity without
having to go into half points.
Time Sensitive Metrics
I suggest that your CEM should be calculated periodically (for example,
monthly) as tracking changes in the metric is a great indicator both at the
tactical level (to understand if specific actions made a difference) and at a
strategic level (to track overall engagement for customer segments). In order
for this time dimension to be useful, the component metrics you choose will
have at least some elements that change over time. For example: Number of
logins in the last month, or 3 month Rolling average of Tech Support cases.
A useful formula hack to convert single entry metrics into time sensitive
metrics is to build an ageing factor, where the points awarded for any
specific activity decrease based on the age of the activity (count in days, for
example). See more on this on the graduating scores section below.
Bound Subtotals
Having bound subtotals means that each of the component metrics in your
CEM is capped at whatever their maximum weighted value. For example, if
you have decided that your “Product Usage” Category is going to be worth
30% of the score, then you need to design your formulas so that the
maximum possible score for all items in that category (combined) equals 30.
This ensures that the weighting rules apply.
Graduating Scores
As mentioned before, a big part of the value of having a CEM is to be able to
track changes over time. We have covered how to incorporate a time dimension
to make this possible, but it is also important to consider the scale of these
changes and to design your component metrics so that relevant movement in
any of the underlying metrics produces relevant and proportional movement in
your CEM.
What you want to avoid is a scenario where all your component metrics are
binary in nature, meaning they get either 0 points or full points. The sections
below illustrate useful techniques to graduate your component metrics.
Please note that getting to the exact formula that produced the right amount
of movement will be a trial and error process, and is not something you
should expect to get 100% correct on your first try.
Points “for playing”
A great technique to use is to divide your category into 2 asymmetrical parts,
and award points to each part using a different method. For the smaller part,
use a binary approach, and award all points based on the answer to a simple
yes/no question. For the larger part, award the rest of the points based on a
more graduated scale, using one of the techniques listed below.
Category 1
(40%)
Category 2
(30%)
10 points
“for playing”
Remaining 20
Graduated
Category 4
(10%)
Category 3
(20%)
For example, coming back to our Tech Support Component, assume that the
total category is worth 30 points. You could break it up into two and award
the first 10 points simply based on whether the Customer has had any
contact with Tech Support (yes/no question) and award the remaining 20
points based on, for example, the distribution of cases in time.
Using this points-for-playing technique will tend to naturally form clusters of
customers with similar CEM scores, but will still allow you to differentiate
among them. It also provides a nicely graduated scale where individual
components can build towards hitting their maximum score.
7
DIY GUIDE
9. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
By Ageing
Ageing is a great hack for artificially graduating a binary metric. The basic
idea is that you will award points for the desired behavior, but then
decrement them gradually depending on how long ago they happened. Let’s
use product usage as an example:
Imagine your product usage category revolves around users running certain
reports in your product, and it is worth 20 points. You want to know (and award
points for) doing this, but you also will want to award more points for Customers
that have done this behavior more recently. If you know the date when they ran
their last report (LastReportDate), you could build a formula like:
This would divide the total points awarded by the age of the trigger activity.
So points will rapidly decrease for every day since the trigger activity
happened.
A variation of this that would produce a less drastic reduction might
incorporate a “grace period”, during which no points are deducted. The
formula might look something like this:
Notice that in both cases the score will never by 20, and that you get a
graduated curve for both scenarios:
You can experiment with different kinds of grace periods and with different formals to use in the denominator to achieve different effects. I recommend you
always model your formulas in excel starting with very simple scenarios to amake sure they will give you predictable results.
By Comparison to a Standard
There are metrics where the timing of the last trigger action is less
important, and instead you might want to compare a number to a standard.
For example, you might know by experience that an Engaged Customer
typically consumes 50 Gb of storage on your platform. You can create
formulas that produce different graduated scores on this category by finding
a mathematical function that will produces a shape that corresponds to what
you are trying to convey. As always, you will need to make sure you build a
cap into your formula so you don’t go over the allotted weight.
For example, assuming a total weight of 40, your formula could be any one
of the following:
Linear Model: Useful when you want to award point proportionally to
progress from 0 up to your goal.
Logarithmic Model: Useful when you want to weigh your reward towards
the beginning of the scale.
Power Model: Useful when you want to weigh your reward towards the end
of the scale.
All 3 examples provide very different shapes that could be more or less appropriate depending on the type of behavior you are modeling.
8
DIY GUIDE
10. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
By Frequency – or distribution in time
This last type of graduation can get complicated, but the principle is similar:
You are trying to reduce a series of numbers into a single score that
evaluates their distribution and rewards some kind of “shape”.
Uniformity
The most straightforward example would be the scenario where you are
trying to reward uniformity. In that case, you can take advantage of the
Average and Standard Deviation functions in excel and create the following
formula (assume a weight of 40):
This would produce the following scores for each scenario:
Frequency
A variant of the above example would be a function that will reward
recurring activity of some kind, regardless of the volume. In this case the
highest ranked scenario would be one where there is any activity (high or
low, but 0) on each period. Again, assuming a Weight of 40, you could
use the following formula:
Where #ofPoints is the number of data points and PointRange is the excel
range where the values are stored. This would produce the following scores
for each scenario:
Modifiers
Now that you have formulas for calculating each of your composite metrics
scores, the total CEM formula will look something like:
This is the simplest case, but depending on how you set up your metrics and
whether or not you have different types of customers, you might need to
adjust your CEM to keep the metric consistent across your customer base.
For example, imagine you have both direct and indirect Customers, and your
indirect customers do not have access to your Tech Support function (as this
is delivered through partners). If you have built a CEM that included a Tech
Support Component worth 20 points, unless you exclude that element for
indirect customers you would be unfairly penalizing them when compared to
your direct customers. If that was the case you would modify your CEM to
include a conditional statement and a different base:
Finally, you might want to include a global modifier to your CEM to include a
factor that is not easily represented as a Component Metric. A good example
of this might be semi-qualitative information coming from your Customer
Success team as a direct result of contact with the customer. If, say, you
have evidence that a given customer is very engaged despite what their stats
say, you could include a set of bonus points that apply to the overall CEM
score. To maintain the base 100 aspect you can build in a cap as follows:
This would be a good strategy to use to give your new customers a grace
period of X months, to avoid raising false alarms due to Customers getting
ramped up.
9
DIY GUIDE
11. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
USING THE CEM IN PRACTICE
Now that you have a CEM you can use to benchmark your customers’ engagement levels through time, you will want to make it available to the teams within
your organization that can take advantage of this new tool. The following illustrates the most common use cases.
Area CEM Users
Customer Success Team Prioritizing mechanism (Which customers to reach out first)
Early warning system (Identify churn risk)
Measuring the team impact (before-and-after CEM)
Account Management / Renewal Sales Identify True-up / Up-sell opportunities
Proof points to demonstrate usage / value of the service
Early warning system (Identify churn risk)
Prepare for difficult conversations with un-engaged customers
Marketing Identify potential “evangelists” for your service
Product Management Monitor adoption rates
Senior Leadership As an overall indicator of customer engagement
»» By Segment
»» By Cohort
»» By Vertical …
As a predictor of growth / churn
To inform investment decisions (internal and external)
A big factor that will determine how much value you get out of your CEM is how you make it available with your organization. Ideally, you will have a way of
getting this value back into your CRM system, as that is the place where people interacting with your Customers are already living.
This is possible even if you used the techniques outlined in the document and calculated your CEM in Excel. Most CRM systems will allow you to create
custom fields and/or objects and bulk upload data points. If you are going down this path, consider creating a new custom object that can store several CEM
entries per account (so you can track history). In addition, you might want to also score the values of the individual Component metrics alongside your CEM,
so users can go one level down and understand how each customer CEM is calculated and what changed over time.
Feeding your CEM back into your CRM system will accelerate its adoption and open up all sorts of sharing opportunities and further analysis via reports and
dashboards, so this is very much recommended even if it requires a few extra hours a month to update.
10
DIY GUIDE
12. Learn more about what
your organization can
accomplish with ACL.
Call 1-888-669-4225 to speak
with a representative
Visit our website at acl.com
Email us at info@acl.com
Customer-Engagement-en
ICE: ACL’S CUSTOMER ENGAGEMENT METRIC
As I mentioned in the introduction, I developed this methodology while designing and implementing ICE, ACL’s Customer Engagement Metric. Below is a
summary of the main elements that make up ICE and some of the insights we have generated through it.
Index of Customer Engagement
Calculation: ICE= Max ( 100 , Base Score + NPS Component )
The Base Score can range from 0 to 100 and ICE can range from -40 to
+100. The NPS component only affects promoters or detractors. Neutral
NPS scores do not impact the ICE Score.
Activation
(40%)
Viral GRC
(30%)
DetractorPenalty(-40pp)
PromoterBonus(+20pp)
On time
Renewal (10%)
Tech Support
(20%)
Ticketing System NPS Component
OR
Note the base 100 score and the use of the asymmetrical NPS modifier.
Each of the elements of the ICE base score except for On Time Renewals
use the points for playing concept:
■■ Activation: 10% for first user activated and .33% per each % user
activated.
■■ Viral GRC: 10% for any use and remaining 20% awarded using a linear
ageing function with a grace period.
■■ Tech Support: 10% for any cases during the last rolling 10 months.
Remaining 10% awarded using the frequency function
The NPS component is a modifier that only affects customers that have
submitted an NPS survey.
Below are a few examples of the individual scores, pure score (from the base
only), and ICE Score (that incorporates the NPS factor) for two groups of
accounts: Very engaged accounts and “On the limit accounts”.
ICE Examples
Most Engaged Accounts “On the limit” Accounts
Even though ICE is currently calculated outside of our CRM, we have created a custom object and summary fields in Salesforce.com to provide visibility for
anyone that interacts with customers as well as to drive reports and dashboards:
11
DIY GUIDE
