O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
olark.com
Measuring Chat Success
Presented by Karl Pawlewicz, Voice of Olark
February 7, 2017
@karlpawlewicz | @Olark
olark.com
Agenda
What does it mean to be a Data Driven Business?
What's in the Olark data stack?
How are we organized to m...
olark.com
What does 'data driven' mean?
Leadership/CEO
"Data has been an essential part of business since the existence of...
olark.com
What does 'data driven' mean?
Marketing
"Using data to make better team decisions, I can see which campaigns are...
olark.com
What does 'data driven' mean?
Product/Research
"The way I see it, data is most useful when it helps you identify...
olark.com
What does 'data driven' mean?
Engineering
"- project requirements include identifying / collecting / analyzing k...
olark.com
What's in the Olark data stack?
Customer data:
Looker
Segment
Amplitude
Google Analytics
Marketing data:
Hubspot...
olark.com
How are we organized?
Team size:
Data department of one consults with all departments to understand common and
t...
olark.com
Data from chat?
Quantitative:
Chat volume
How's my team doing
Revenue
Where are people chatting on my site
What ...
olark.com
Data from chat?
Quantitative - Volume
olark.com
Data from chat?
Quantitative - How's my team doing
olark.com
Data from chat?
Quantitative - Revenue
olark.com
Data from chat?
Quantitative - Where are people chatting on my site
olark.com
Data from chat?
Quantitative - What percent of people chat
olark.com
Data from chat?
Quantitative - Where country are visitors chatting from
olark.com
Data from chat?
Qualitative -
Product feedback
olark.com
Data from chat?
Qualitative - How are people
comparing me to my competitors?
olark.com
Data Driven Decisions IRL
What do you want chat to do?
Improve customer happiness?
How to measure: Chat reports,...
olark.com
Data Driven Decisions IRL
OnePageCRM -
Alan O'Rourke at OnePageCRM:
Manually compared new customers in their CRM...
olark.com
Upper90Soccer.com -
Ben Jata of Upper90
"Should I add more
people on chat?"
Conversion Rate increased
11% when c...
olark.com
Data Driven Decisions IRL
AcquireConvert.com -
Giles Adam Thomas of AcquireConvert:
Use the same language your c...
olark.com
Data Driven Decisions IRL
Disruptive Advertising -
Jacob Baadsgaard of Disruptive Advertising:
Tested four 'unex...
olark.com
Want to learn more?
1. Pandas, scipy, scikit-learn (python libraries) are all well documented
2. reddit.com/r/da...
olark.com
One last thought on data people...
1) Domain Expert
You need someone with domain expertise on you data team. The...
olark.com
Thank you! Questions?
Karl@Olark.com
Olark Data Scientist: Bens@olark.com
@karlpawlewicz | @Olark
Próximos SlideShares
Carregando em…5
×

NYC Data Driven Business Meetup - 2.7.17

158 visualizações

Publicada em

An overview of the quantitative and qualitative data provided by live chat, and how to measure the sales, marketing, and customer support ROI of a chat widget.

Publicada em: Tecnologia
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

NYC Data Driven Business Meetup - 2.7.17

  1. 1. olark.com Measuring Chat Success Presented by Karl Pawlewicz, Voice of Olark February 7, 2017 @karlpawlewicz | @Olark
  2. 2. olark.com Agenda What does it mean to be a Data Driven Business? What's in the Olark data stack? How are we organized to make data accessible? What kind of data does chat (or a similar customer comms channel) provide? Real world examples of data driven decisions Where can you go online or IRL for more insight on data best practices? Q&A
  3. 3. olark.com What does 'data driven' mean? Leadership/CEO "Data has been an essential part of business since the existence of business. The main difference is now it's easier to get both good and bad data. To be data driven is not to ignore hunches and intuition, but to figure out the best method to validate your ideas, and to act on it. "Data drive decision making reduces uncertainty, but does not remove it. Without data, we are just guessing, and hoping for the best." "Assess our performance at setting and achieving objectives. "Help product managers make decisions to determine where to focus their efforts, and understand the results of our efforts in the past, and help frame the revenue impact." "Deciding where to invest in growth, I use information from inside and outside of the organization to make informed decisions on the potential impact of various options."
  4. 4. olark.com What does 'data driven' mean? Marketing "Using data to make better team decisions, I can see which campaigns are working (or not), and easily calculate ROI - not just for dollars spent, but also for the time and resources we're investing into each activity. Being data-driven, especially in marketing, makes it easy to think strategically, plan better, help our own team understand how their contributions affect our goals, and show the company as a whole how we're impacting the bottom line."
  5. 5. olark.com What does 'data driven' mean? Product/Research "The way I see it, data is most useful when it helps you identify "obvious" and inarguable truths i.e. "Wow what a huge difference in performance, we should definitely do that thing" or "Wow that's clearly a bad idea, it got dominated by this other thing." Data helps provide definite answers for smaller, simpler problems. But sometimes, for bigger and more difficult decisions that seem to be 50/50 even after you've seen all the data, those are the ones where great intuition come in and (after a certain point) no amount of additional data would help. Steve Jobs is a common example of someone with great product insights. No amount of data would have been able to definitively tell him either way that a new device like the iPad would be a huge success. However, I'm sure data allowed Apple to become pretty confident in taking that risk."
  6. 6. olark.com What does 'data driven' mean? Engineering "- project requirements include identifying / collecting / analyzing key metrics on which to base decisions - project deliverables include some report on those metrics More generally, projects are seen as experiments, and therefore are not complete without the data and the analysis."
  7. 7. olark.com What's in the Olark data stack? Customer data: Looker Segment Amplitude Google Analytics Marketing data: Hubspot Unbounce SEMRush Moz Google Analytics Third Party APIs Monkey Learn Builtwith Alexa FirstOfficer
  8. 8. olark.com How are we organized? Team size: Data department of one consults with all departments to understand common and team-specific needs. Important aspects of this system: One central warehouse so that all team members are using the same data. It's kept up to date and is known to be high-integrity. Data accessibility in incredibly important, but having standard definitions that everyone on the team knows and can trust is as important. If multiple team leads are building reports, it helps to have the data team define and implement the metric definitions so that everyone is computing the same thing, and not people reporting metrics with the same name differently.
  9. 9. olark.com Data from chat? Quantitative: Chat volume How's my team doing Revenue Where are people chatting on my site What percentage of people chat What countries are visitors chatting from Qualitative: Product feedback What are people saying about my company (sentiment) What are people saying about my competitors Which chats lead to sales
  10. 10. olark.com Data from chat? Quantitative - Volume
  11. 11. olark.com Data from chat? Quantitative - How's my team doing
  12. 12. olark.com Data from chat? Quantitative - Revenue
  13. 13. olark.com Data from chat? Quantitative - Where are people chatting on my site
  14. 14. olark.com Data from chat? Quantitative - What percent of people chat
  15. 15. olark.com Data from chat? Quantitative - Where country are visitors chatting from
  16. 16. olark.com Data from chat? Qualitative - Product feedback
  17. 17. olark.com Data from chat? Qualitative - How are people comparing me to my competitors?
  18. 18. olark.com Data Driven Decisions IRL What do you want chat to do? Improve customer happiness? How to measure: Chat reports, NPS, Increase conversions? How to measure: Number of conversations, Number of sign-ups Increase sales? How to measure: Revenue reports, CRM results Product feedback? How to measure: Chat transcripts Marketing insight? How to measure: Chat transcripts
  19. 19. olark.com Data Driven Decisions IRL OnePageCRM - Alan O'Rourke at OnePageCRM: Manually compared new customers in their CRM with Olark logs to see who talked to customer success before signing up. New customers who chatted to customer success before signing up were almost twice the size of our average customer. His two person live chat team brought in an additional $24,000 in new business in over 12 months (not including renewals). 240 hours on chat, earning $100 every hour - an average that out performed most of their other marketing channels. Based on this data, Alan made the decision to hire another person to staff chat on OnePageCRM's marketing site in a timezone they weren't already covering. SOURCE: https://blog.olark.com/how-to-measure-live-chat-success-with-analytics
  20. 20. olark.com Upper90Soccer.com - Ben Jata of Upper90 "Should I add more people on chat?" Conversion Rate increased 11% when chatting Full-time chat specialist generated $40K in revenue in roughly 6 months Jan 16 Feb 16 Mar 16 Apr 16 May 16 Jun 16 Jul 16 Aug 16 Sept 16 Oct 16 Nov 16 Dec 16 OLARK Total chats 5 4 10 7 7 154 328 407 391 416 431 428 Great chats 2 1 4 0 1 24 45 54 82 48 46 10 Chats to review 0 0 0 1 0 3 5 0 6 4 3 1 Average chats per day 1.67 1.33 2.5 1.4 1.75 8.56 13.12 15.65 17.77 16.64 17.96 22.53 Median initial response time (seconds) 16 13 30 18 24 28 25 24 19 19 18 23 Offline messages 23 54 46 36 38 55 59 55 64 55 73 64 Ratings (5 stars max) Overall chat 4.5 5 4.75 3 5 4.85 4.63 4.78 4.75 4.8 4.86 4.55 Responsiveness 5 5 5 4 5 4.83 4.67 4.84 4.83 4.74 4.89 4.91 Knowledge 5 5 4.75 5 5 4.64 4.57 4.74 4.67 4.67 4.71 4.55 Friendliness 3 5 5 3 4 4.79 4.7 4.8 4.89 4.7 4.84 4.91
  21. 21. olark.com Data Driven Decisions IRL AcquireConvert.com - Giles Adam Thomas of AcquireConvert: Use the same language your customers use to increase conversions Copy transcripts to a spreadsheet Evaluate for commonly used words and or phrases Insert those phrases into your landing page and CTA copy "We saw a 176.33% conversion rate increase through this copy change alone." SOURCE: https://blog.olark.com/how-to-use-transcripts-to-increase-conversion-rates-and-profits
  22. 22. olark.com Data Driven Decisions IRL Disruptive Advertising - Jacob Baadsgaard of Disruptive Advertising: Tested four 'unexpected' greeters on the homepage for their business, a PPC ad agency: “What’s your best marriage advice?” “If your AdWords account was an animal, what would it be?” “Who’s your #1 competitor?” “What are you struggling with right now?” Winning test was _____________ Also tested on on a client who sells promo products for lanyards: “How many lanyards are you looking for?” “Hi! I’m Brad! Let me know if you need any help!” “What event do you need the lanyards for?” The Greeter that won by a landslide was “How many lanyards are you looking for?” Using the Olark Greeter in clever ways on clients' landing pages increased conversion rates by as much as 37 percent! SOURCE: https://blog.olark.com/how-clever-greeters-increase-conversion-rates
  23. 23. olark.com Want to learn more? 1. Pandas, scipy, scikit-learn (python libraries) are all well documented 2. reddit.com/r/datasets/ is a fun place to find datasets to teach yourself things 3. UMcoursera course starting soon: www.coursera.org/learn/python-data-analysis 4. Books: a. https://www.amazon.com/Python-Machine-Learning-Sebastian- Raschka/dp/1783555130/ b. https://www.amazon.com/Data-Science-Business-Data-Analytic- Thinking/dp/1449361323/
  24. 24. olark.com One last thought on data people... 1) Domain Expert You need someone with domain expertise on you data team. The particular domain obviously depends your data teams role. Here, it's mostly company health and customer behavior. If your data team is focused more on your product (e.g. fitness monitor company has data team to do something interesting with your heartrate time series) then people you hire should have some exposure to that or a similar domain and exhibit the ability to learn about it independently. 2) Researcher Design experiments, execute them, make sure the results are statistically valid. Compromise between business goals and timelines and confidence (not something you have to do in academic science). Doubt everything. 3) Computer Scientist Making computers do stuff. Know how to optimize queries, use efficient computation patterns. Maintain production code, design and or use complex algorithms. If you're team does a lot of ML, this expertise can run very deep (not the case here). A person should be able to read about how an algorithm works, understand how to use it, it's limitations, red flags, etc. It's a bad sign if someone just williy-nilly applies techniques without checking that the premise of the technique holds, and assumes all results are valid. 4) System Administrator I manage our data warehouse. Many tasks are similar to that of a database administrator. I need to make sure our data is secure, the databases are optimized for the types of queries we're doing, they're up, the data is correct, design the schema, etc. Security is pretty easy, since it's not customer facing and there are relatively few users. This may not be the case at larger companies. Eng ops also helps and hand holds this a bit, same with keeping the database up and running. Verifying and cleaning the data is awful and time consuming. "data munging" is the thing that every data person spends more time doing than they would like. SOURCE: https://www.aapor.org/AAPOR_Main/media/Task-Force- Reports/BigDataTaskForceReport_FINAL_2_12_15_b.pdf
  25. 25. olark.com Thank you! Questions? Karl@Olark.com Olark Data Scientist: Bens@olark.com @karlpawlewicz | @Olark

×