“Big Data: Leveraging Competitive Intelligence In Retail" focused on the next wave – enabling real time decisions and real-time responses through big data. Here are the Lounge47 key takeaways: 1. Large enterprises have this far used big data to focus on process improvement and variety of data (Process improvement 47%, Variety of data 26%, Volume of data 16%, Cost Saving & Efficiency 8%, Velocity of Data 3%) 2. Big data is not a new problem; at any point of time, our ability to produce data has always been greater than the sophistication of the tools available to process and make it usable 3. Companies like Uber and Amazon, with products like “Surge Pricing” or “Dynamic Pricing” are ushering in the paradigm of “fast data” to make instant decisions and gain a competitive advantage 4. “Fast Data” unlike “historical data, is live, interactive, automatically generated, and often self-correcting” – the volume and nature will be further accelerated through the Internet of Things (IoT) 5. In the retail vertical – data enablers that push micro decisions in real time and serve to answer – what inventory to hold? or what products to promote? - pose a powerful value proposition 6. A plethora of data products, web-based, Apps, API’s, reports could be built to help enterprises take decisions E.g. a “Color” report that tells a fashion retailer that their inventory should carry more items in blue 7. Data products could serve - ecommerce companies, sellers, brands – each stakeholder, with very specific requirements and specific problems to solve E.g. brands value reports on product discounts offered to flag policy violation 8. Solving the big data challenge would involve the following generic steps – data extraction and aggregation, cleaning, normalizing, standardizing, sorting, storing. Analytics. Visual data presentation, via dashboard interfaces, reports etc. 9. Big data sounds like a simple problem to solve however the challenges are many a) Data acquisition: crawling public websites could be limited if volume and speed of query impact service to users, thus slowing the data collection b) Data cleaning & standardization: raw data could be messy or have gaps c) Storage and retrieval d) Data Accuracy: Careful management of massive machination with minimal human audits to keep the margin of error suppressed 10. Some Big data products: Price comparison by the hour and across competition, color report on product inventory, Market & Business intelligence products, discount tracking of basket of products 11) Finding a “give-back” to encourage E-Commerce companies to part with private data would allow big data companies to build an ecosystem that is mutually beneficial to all stakeholders. While big data is an often used buzz word, and challenges like “new technology deployment” and the “collection, analysis and measurement of data” are being solved, the full power of this paradigm will be realized when organi

1. Mandar R. Mutalikdesai
Head of Data Semantics
mandar@dataweave.in
Data is the new Fabric
COMPETITIVE INTELLIGENCE IN RETAIL

2. BIG DATA AND DATA PRODUCTS
"Big data": publicly available as well as within intranets
Huge opportunities, because data = insight
Need of the hour: data democratization---intra-, inter-, and trans-firewall
Many challenges: aggregating data, cleaning data, normalizing data, learning
patterns and insights, presentation
This is where data products come in!

3. BUILDING DATA PRODUCTS
• Aggregate large amounts of data publicly available on the web, and serve
it in readily usable forms
• Serve actionable data through APIs,Visualizations, and Dashboards
• Provide reporting and analytics layer on top of datasets and APIs
Build data products that provide timely actionable insights for
businesses and consumers by aggregating and analyzing public data on
the web

4. SOME CHALLENGES
Data acquisition: crawling and extraction infrastructure
Data is uncertain and incomplete
Data is human understandable, not machine understandable
Data storage and retrieval
Analytics: more data (often) beats better algorithms
Product UX

5. Pricing Date
Open Government Data
Social Media Data
Attributes
Attributes
Big Data
Platform
Unstructured, spread
across sources and
temporally changing
JSON, PDF, XML,
DOC, XLS, HTML and
much more
Data APIs
Dashboards
Data Services
API Feeds
Visualizations and
Widgets
DATAWEAVE PLATFORM

6. AGGREGATION AND EXTRACTION
Extraction Layer: Offline Extraction of Factual Data
Aggregation Layer: Distributed Crawler Infrastructure
Public Data on the Web

7. NORMALIZATION
Nomalization Layer
Extraction Layer
Offline extraction of factual data
Remove Noise Fill Gaps in Data
Represent Data Clustering
Machine
Learning
Techniques
Knowledge
Base

8. DATA STORAGE AND SERVING
Serving Layer
Indexes Views
Filters Pre-Computed Results
Highly
Responsive
Serving Layer
Distributed Data Storage
Crawl Snapshots
Processed Data
Clustered Data
DashboardsVisualizationsData APIs Reports

9. RETAIL ANALYTICS
eCommerce Big Data made easy!
Data science made easy for every retail/brand manager
Make better decisions on pricing, competitive strategy, product coverage
with full visibility.

10. PRICEWEAVE
Real-time Retail Insights Platform across Geographies
Pricing and Assortment Analytics Product and Catalog Intelligence
Price Monitoring,
Availability & Historical
Analysis
Gap Analysis & Coverage
Along Multiple
Dimensions
Product Similarity Based
on Multiple Attributes
Catalog Spread & Depth
in Key Product
Categories

11. PRICING INTELLIGENCE
Compare and monitor product prices, availability,
offers across competitors in real-time
• Improve margins by responding to pricing opportunities proactively
• Increase sales through competitive offers, promos etc. (e.g. free shipping)
• Drive price parity across distribution channels. Detect pricing violations.
• Develop long term strategies with pricing trends & insights from the past

12. ASSORTMENT INTELLIGENCE
Benchmark and analyze your SKU coverage to
identify gaps and strengths in your catalog
• Increase coverage by focusing on a wider set of customers
• Identify & promote products, brands, categories where one has a unique
advantage
• Discover product or category gaps in catalog
• Reduce customer acquisition costs

13. PRODUCT SIMILARITY MAP
Compare products based on features, tech specs, pricing and
brand attributes. Identify similar and substitute products
• Fulfil customer demand for a given product with a compelling offer on a
substitute to gain share
• Cover out-of-stock products with similar ones
• Compare competitor coverage, regardless of SKU similarity

14. COLORVARIANT ANALYSIS
Decipher trends in colors, patterns, and variants for
apparel, footwear and lifestyle products. Use colors,
fabric, and style for assortment analysis.
• Spot current and predict future trends in colors and patterns
• Ensure most popular colors and sizes are stocked in line with demand
• Gain insights from social channels about brands, clothing trends

15. PROMOTION ANALYSIS
Promote the right products. Enhance stock availability of
popular products. Understand competitor promotion strategy.
• Gain understanding on products that are being featured by competitors
• Discounts, coupons and price points at which the promoted products
are being offered
• Get insights into competitors’ product rankings and promotions on a
ongoing basis
• Ensure the products customers are interested are always in stock and
attractively priced

16. ALERTS AND NOTIFICATIONS
Set alerts on any attributes that need tracking, in real time. Get
informed on new product introductions, hot deals, and price
slashes, as they happen.
• Get insights in to competitor’s strategy
• Discover pricing violations & unauthorized discounts on your brand
• Keep tabs on the market trends, in real time

17. CUSTOM REPORTS
Generate custom reports pivoted around: SKU, sub-
category, category, brand, price, geography and
other attributes
• Pick your priorities to pivot the reports & gain actionable insights
• Understand key trends relative to products, brands, categories and
geography
• Point-in-time snapshots to understand market offers for product/
category
• Compare snapshots for trends / gaps

18. HOW IT WORKS
Data Acquisition
Deep Web crawling
• Daily update of 12
million product prices
• 50 million data points
monitored
• 3TB of data processed
on a daily basis
Data Extraction and
Normalization
• Extract relevant data
points
• Clean data stored
• Normalized records
based on knowledge
bases
• Noise cancellation
Machine Learning and
Information Retrieval
• Classify products into
product types
• Clustering same
products (& similar
products)
• Multi parametric
clustering
Data Representation
• Dashboards
• Visualizations
• APIs
• Alerts and
Notifications
• Reports
• Integration

19. THANKYOU