Enterprises are investing a lot of time and money into unlocking the value of Big Data for business and competitive intelligence – but are they overlooking the richest source of data available today?
In this deck, Connotate shares the fundamentals of collecting data from the Web to capture insights into competitors’ pricing, product positioning and consumer sentiment to fuel better decision-making.
Topics covered include:
- Basic differences in data sources (surface Web, hidden Web, automated log in, social media)
- How to compare cost/benefits of manual versus automated approaches
- Guidelines for making build versus buy decisions
-The bottom-line impact of data quality and accuracy
- Options for refining data to obtain business insights
WordPress Websites for Engineers: Elevate Your Brand
Big Data and Competitive Intelligence
1. Big Data and Competitive Intelligence
Finding Answers Outside the Firewall
Presenters: Isai Shenker, VP of Product Management, Connotate
Gina Cerami, VP of Marketing, Connotate
Moderator: Christian Giaretta, VP of Sales Engineering, Connotate
Date: September 20, 2012
2. Presenters
Isai Shenker Gina Cerami
Vice President of Product Management Vice President of Marketing
Connotate Connotate
3. Today’s Discussion
• Competitive pricing: Insight from outside the firewall
• Business case and operational challenges
• Case study: Competitive pricing for online retail
• Case study: Pricing optimization for specialty chemicals
• Customer sentiment: Digging into Deep Web
• Sources of consumer sentiment
• Process: Extracting consumer sentiment from YouTube
• Case Study: Allergy medicine and consumer sentiment
• Manual versus automated approaches
• Q&A
5. The Business Case
Distributors and resellers prices
15,000,000 prices
Other competitor’s prices
6. Operational Challenges in Setting Prices
Measuring the impact
of pricing decisions
Forecasting the impact of
potential pricing decisions
Getting
Maintaining visibility into
access to
promotional profitability robust,
Getting access to competitive
robust competitive price data
price data is a
Making sure that stores change
prices accurately and timely big challenge
Keeping up with promotional deals Retail Systems
cut by buyers with manufacturers
Research 2012
Managing promotions Benchmark Report
across channels
7. Polling Question
Are you currently collecting competitive pricing data
from the Web?
Yes – we are collecting pricing data from the Web
No – we are not collecting pricing data from the Web
but we have a project or plan to do this within the next
6 months
No – we have no plans to collect pricing data from
the Web
8. Competitive Pricing:
Real-World Success Stories
Online Retailer of Specialized Retailer
National Store Chain Goes Deep Web
New and Used Goods
• Checks 1M online • Checks prices on • Builds online
prices per day to 3M items at catalog by
win business national and local going 3-7 layers
levels deep on direct mfrs’
Web sites for
product info
Automation is key to success
9. Competitive Pricing:
Real-World Success Stories
Online Retailer of Specialized Retailer
National Store Chain Goes Deep Web
New and Used Goods
LOWEST PRICE OPTIMIZED DOUBLED SIZE OF
MARGINS ONLINE CATALOG
Automation is key to success
10. An Overview of the Automation Process
Connotate
Partners
{
{
Collect Data Transform Deliver
Internal Sources External Sources • Structure • Reports
• Database • Social Media • Classify • Dashboards
• Market Basket • Surface Web • Prep for Analysis • Workflow
• Inventory, etc. • Hidden Web • BI Plug-ins
•Secured Sites
12. Price Optimization Case Study:
Sigma Aldrich
Sigma-Aldrich faced a “Big Data” challenge
• An overwhelming volume of pricing information on the Web
? $
Jeopardizing
Manual Inaccurate a Sound
Processes Results Pricing
Strategy
16. Customer Sentiment
“The better and more quickly a company
can measure customer sentiment,
the sooner it can tell how well a product
is likely to sell.”
– BusinessWeek, 2011
17. Customer Sentiment: Deep Web
A Fortune 50 consumer product goods company sought input
from social media on customer sentiment about a popular
class of products:
• Twitter was easy to access via API but not useful; there was
no in-depth discussion of features
• Detailed product reviews existed on Amazon, Walmart, in
microblogs and comments on YouTube videos
• The firm wanted to access these sites and draw conclusions
from this data to create a feedback loop for their continuous
product design cycle
18. Differences in Web Sources
Facebook
Twitter, etc.
D
A
T
Surface Web, Other A
Social Media
A
Processing
N
Deep Web A
(traverse links)
L
Y
S
Password- I
Protected Sites
S
Data Sources Analysis
18
19. Automation Opens Access to Deep Web and Secured Sites
Facebook
Twitter, etc.
D
A
T
Surface Web, A
blogs, forums
A
Processing
N
Deep Web A
(traverse links)
L
Y
S
Password I
Protected Sites
S
Analysis
Data Sources
19
20. Obtaining Customer Sentiment from YouTube
Manually search YouTube for <“product name”> <“review”>
Use the Connotate automation package to find
<“product name”> <“review”> in posted comments
Use Connotate to follow links
to other sites with <“product name”>
Feed input into analytical engine to reveal sentiment
Graphical User Interface/Presentation of Insights
21. Refining Data for Business Insights: Customer
Sentiment
A major pharmaceutical company wants to understand the
competitive landscape for allergy medications.
• Decided to focus on data available online
• Used Connotate to collect product information, prices and
4,700 customer reviews on 465 products
• Analyzed data and found insights using Luminoso text analytics
• Time from idea to initial findings: 6 hours
25. Another Look at the Automation Process
Your internal BI tools or
Connotate Connotate Connotate partners
Collect Data Transform Deliver
• Reports
Internal Sources External Sources
• Classify • Dashboards
• Database • Social Media
• Structure • Workflow
• Market Basket • Surface Web
• Prep for Analysis • BI Plug-ins
• Inventory, etc. • Hidden Web
•Secured Sites
Connotate provides precise quality data, structured
for delivery to your analysis and presentation tools.
Connotate maximizes the value of your investment
in business intelligence, text analytics and semantic
analysis tools.
Excel
26. Polling Question: Customer Sentiment
Are you currently collecting customer sentiment data
from the Web?
Yes – we are collecting customer sentiment data
from the Web
No – we are not collecting customer sentiment data
from the Web but we have a project or plan to do this
within the next 6 months
No – we have no plans to collect customer sentiment
from the Web
27. Manual versus Automated Approaches
Your Data Needs To Automate or Not?
? May want to consider
Complex product-matching tasks
crowd sourcing
Small amount of data, needed a few ? A manual approach may
times per year suffice
Specific external data (under $5K/year) ? Purchase from 3rd party
High volume data monitoring Automate
Variety of sources Automate
Frequent updates and/or monitoring Automate
Need for data post-processing Automate
28. Vast Resources of Competitive Intelligence are
Outside the Firewall
Automation is Price
the key to optimization
extracting
precise quality Product design
data
Deliver new
online services
Automation is
needed when
Dynamically
the initiative is
adjust product
strategic
positioning
29. Selected Connotate Partners for
Competitive Intelligence
Text and Sentiment Analysis
Data Integration and
Pricing Intelligence CrowdSourcing
Business
Intelligence
Special thanks to Nikki Baird,
Retail Systems Research,
www.rsrresearch.com
30. Q&A
Connotate will email a link to this presentation as well as a
copy of the slides to you within 2 business days.
If you have an immediate need and would like us to contact
you about a forthcoming project, please check the appropriate
box in the last polling question or call us at 732-296-8844.
You may also visit www.connotate.com for more information.
31. Thank You
If you have an immediate need and would like us to contact
you about a forthcoming project, please check the appropriate
box in the last polling question or call us at 732-296-8844.
For more information, visit www.connotate.com.