The document discusses making text analytics from experience data actionable by balancing human and AI capabilities. It describes how organizations hold large amounts of unstructured data and outlines key aspects of text analytics like categorization and sentiment analysis. The main points are that AI alone is not enough and a hybrid human-AI model is needed, where human expertise ensures precision, business alignment and explainability that AI lacks. It provides a 7-step checklist for success with text analytics programs that includes understanding needs, planning frameworks, guiding AI modeling, auditing results, taking action on insights, and maintaining the system.

1. Making Experience Text Analytics
Actionable: The Human/A.I. Balance
Alison Bushell & Jason Bryce
Confirmit
The Festival of NewMR, March 2020

2. Sponsors
Gold
Silver
The Festival of NewMR, March 2020

3. Alison R. Bushell | Director, Analytics
Jason Bryce | Sr. Analytics Consultant
March 17, 2020
Making Experience
Text Analytics Actionable:
The Human/AI Balance

4. § Structured v unstructured data
§ Where is this free-form text
data held?
§ Organizations are holding over
100 terabytes of unstructured
data!
Data Overload

5. Key Ingredients
Categorization Sentiment
+

6. Text Analytics
“Tried ordering on your website it wouldn’t accept the offer
code which was frustrating enough, then I called and spoke to
your agent Julie who was very pleasant and professional but
could not fix the problem either. Not impressed.”
Order Web Site Staff Sentiment
Not impressed
frustrating
very pleasant
professional
ordering
offer
your website your agent
code

7. § Categorizing your data can be a
daunting task
§ Organizations are turning to AI
powered technologies to speed
up the process
§ AI isn’t a solution on its own
Getting to Grips with Your Data

8. AI or Human?
Approaches Advantages Disadvantages
A.I. (automated) § Saves time
§ May identify previously
unknown topics
§ Low Precision
§ Not actionable/not business-aligned
§ Not easy to explain nor
troubleshoot
Rule-based (human) § High Precision
§ Actionable/business-aligned
§ Easy to explain and
troubleshoot
§ Requires human expertise
§ Not as quick
§ Risk of Overfitting
Hybrid
Maximizes advantages and minimizes disadvantages

9. A.I. + Human Categorization Modelling

10. Getting Emotional – Categorization Model
Awareness Purchase Service Claim Renew

11. 7 steps for you to take away
and think about for your text
analytics programs
Your Checklist for Success

12. Key Points:
§ Hard lessons are expensive
§ Competencies are available
to help you succeed
1. Don't Reinvent the Wheel
¨ Read up on it (thoroughly)
¨ Talk to experts
¨ Get help

13. 2. Do Your Due Diligence
¨ Meet with stakeholders and understand
their needs
¨ Get buy-in
¨ Plan forward. Manage backward.
¨ Understand your data and where it's
coming from
¨ Understand what A.I. can and cannot do
What to Understand:
§ Journey maps
§ Org charts
§ Strategic and operational plans
§ KPIs and company metrics
§ Pain points
§ Initiatives
§ Corporate goals/mission/vision
§ What executives are on the hook
for
§ More…

14. Newsletter
Blog
Inserts
Social Media
Text Alerts
E-mails
3. Draft a Framework Examples
Operational Strategic Customer Journey
Product
Dining
Staff
Payment
Communications
Retention
Niche Targets
Marketing
Pricing
Innovation
J1 Awareness
J2 Onboarding
J3 Servicing
J4 Claims
J5 Renewals
Interactions
Products
Services
Processes
Documentation
Etc.
Loyalty
Brand Love
Recommending
Suggestions
Year 2+ benefits
Etc.

15. 4. Make the Most of AI Topic Extraction
Guide Refine Solo
(Unguided)
Tell it what specific
topics or concepts
to build around
Use Boolean to
add, remove or
combine topics

16. 5. Audit & Test
¨ Scrutinize uncategorized bucket
¨ Audit
¨ Meet with stakeholders again after results are available
¨ Are the categories working?
¨ Are results too detailed, not detailed enough?
Test
Audit

17. 6. Reporting & Action
Reporting:
¨ Metadata background
¨ Impact Analysis
¨ Emerging Trends
¨ Root Cause
¨ Segmentation
¨ Data Source Differences
Action:
¨ Real-Time Alerting
¨ Tactical Action
¨ Strategic Action

18. 7. Maintain & Update
Maintain and Update

19. Thank You
Alison.Bushell@Confirmit.com | Director, Analytics
Jason.Bryce@Confirmit.com | Sr. Analytics Consultant

20. Q & A
The Festival of NewMR, March 2020
Ray Poynter
NewMR
Alison Bushell
Confirmit
Jason Bryce
Confirmit

21. Sponsors
Gold
Silver
The Festival of NewMR, March 2020