SlideShare uma empresa Scribd logo
1 de 36
Baixar para ler offline
How we leverage the cloud,             big-
 data, social data to do BI?



            Salon BI 6 November 2012

      francis@qmining.com
          Founder & CEO
Hidden Agenda
●   Story
●   QMining
●   BI landscape - Issues
●   How QMining is leveraging:
    ○   Open source
    ○   Cloud
    ○   Hadoop for Big data
    ○   Social data
● Innovation
    ○ QMiner
    ○ QMarketing
Story
Products and Consulting




Big Data   Knowledge   Action
Big Data        Predictive Analysis


                Patterns/clusters


             Any data including
Knowledge   web traffic, social data,
                 e-commerce
                 transactions




Action
BI Landscape - Issues



● Frustrating
  ○ not simple/not fast/costly
● Increasing data & sources
● User want more power (bypass IT)
QMining



● Simple/fast/cost effective
● Scalable
● More power to the users (bypass IT)
Why?
Examples

Context                    Enterprise          QMining
  Big Player (content)   18 mois = failure 3 months = success

online marketing         180+ days = not   11 days = success
                              done
online KPI                  30+ days         2 days (bots)
                          investigations
IBM SPSS                     15 days             1 day
                                ...               ...
Where?
Applications
                          Online
                         Marketing
                         ex: Marketing
                      Management Platform




IT Application
Management                                       payment
 ex: performance
   monitoring &
                     QMining                   E-commerce
                                             ex: merchant retention
 troubleshooting




                        IT Service
                       Management
                      ex: business process
                           automation
How?
Methodology

Principles:
● Open source
● Minimize Abstraction
● Cloud

Tools:
● Python
● Hadoop (for Big Data)
Open Source
Leverage collaboration and openness to move faster




                              -71% of clients don't use open source
All abstractions leak
   Abstract -> Procrastinate!
http://www.aleax.it/pycon_abst.pdf   (Alex Martelli : "Abstraction as a Leverage" )
Minimize A Tower of Abstraction
                Simplify & lower the layer of abstraction




Examples:

● Work on file not BD if possible
● Low level linux command lines (cut, grep, sed etc.)
● High level languages : python
● Minimize # of languages; ex: Hadoop streaming
  (map/reduce=python)
● Prototype on a laptop (atom N450)
Cloud
Leverage IAAS & PAAS infrastructure to build new SaaS
Cloud - PaaS vs IaaS

  Pros                            ●    framework                 ●    less limits
                                  ●    deployment                ●    Scaling
                                  ●    multi-tenant              ●    Pricing
                                  ●    https
                                  ●    log & data
                                       visualisation

  General Score

                                 +20                            -20

Recommendations:
                                     Protot                            Productio
                                   ype                           n
The best of both world might be heroku a Cloud application platform but we haven't tried it
montreal python slides on a comparison (appengine vs amazon, pris & cons for a startup)
QMining in Cloud

Principles:

●   Easy deployment of environments
●   Scalable by design
●   Multi-tenant by design (don't manage multiple versions/instances)
●   REST API by design
the power of matlab for prod
            What                                       How
power
of




PaaS
(Platform as a Service)




Web framework

                    example coradiant : 100X speedup
Big Data
Leverage cloud to process lot of data (hadoop)




We use amazon EMR (hadoop, EC2 & S3)
Innovation
Why enterprises fail at
       innovation? (blog)
1.   Salon BI 2011: Philippe Nieuwbourg M&A
2.   Boston CXO Forum 24 October : Best Practice on Global
     Innovation (IBM, EMC, P&G, Intuit) Exploit vs Explore
3.   Brad Feld (Managing Director at Foundry Group)
     Startup community 29 October Notman house MTL
     Hierarchy vs network
QMiner


1. Simplify real user management
2. Simplify business view management
3. Simplify Persona management
QMiner
     Simplify real user management


 10% of your online users are experiencing
                 slowness

● 100% are located in China
● 75% are using mobile device
● 90% are using iphone

        http://qmining.qminer.com
QMiner & Social Graph
             Leverage the social graph apis



Facebook Social Graph:
http://developers.facebook.com/docs/reference/api/

QMiner Example:
http://fr.qmining.com/facebook/
QMiner
           Simplify business view management


Your are losing $10K/hour:
● 100% are Male
● 75% are using mobile device
● 90% are using iphone
http://qmining.qminer.com/api/clients/qmining/apps/fbtest/ruleset/
QMiner
    Simplify persona management



   None buying customers are:
           93% male
          67% single
      87% high educated


Persona fit = 25%
QMarketing
Simplify online marketing management
Investing in online marketing
and not getting the expected
            ROI?
You can't find a global
solution in this fragmented
market?
QMarketing - Marketing Management
    Optimize your online Marketing investments
Hadoop for Channel Optimization
Non sales Patterns with social data
Summary
●   Story
●   QMining
●   BI landscape - Issues (not simple, slow, costly)
●   How QMining is leveraging:
    ○   Open source with linux, python, github
    ○   Cloud with amazon, google appengine
    ○   Big data with hadoop
    ○   Social data with facebook api
● Innovation
    ○ QMiner Simplify real user, business view & persona management
    ○ QMarketing simplify online marketing management
QUESTIONS




info@qmining.com

Mais conteúdo relacionado

Mais de Francis Piéraut

Big data: Just another barrier of entry
Big data: Just another barrier of entryBig data: Just another barrier of entry
Big data: Just another barrier of entryFrancis Piéraut
 
The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.Francis Piéraut
 
Appengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsAppengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsFrancis Piéraut
 
No BI without Machine Learning
No BI without Machine LearningNo BI without Machine Learning
No BI without Machine LearningFrancis Piéraut
 
easy_install digipy & mlboost
easy_install digipy & mlboosteasy_install digipy & mlboost
easy_install digipy & mlboostFrancis Piéraut
 
Machine Learning empowered by Python April2009
Machine Learning empowered by Python April2009Machine Learning empowered by Python April2009
Machine Learning empowered by Python April2009Francis Piéraut
 
Intro to Machine Learning Enpowered by Python (Montreal Python)
Intro to Machine Learning Enpowered by Python (Montreal Python)Intro to Machine Learning Enpowered by Python (Montreal Python)
Intro to Machine Learning Enpowered by Python (Montreal Python)Francis Piéraut
 
Master Defense Slides (translated)
Master Defense Slides (translated)Master Defense Slides (translated)
Master Defense Slides (translated)Francis Piéraut
 

Mais de Francis Piéraut (10)

Big data: Just another barrier of entry
Big data: Just another barrier of entryBig data: Just another barrier of entry
Big data: Just another barrier of entry
 
The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.The big data dead valley dilemma and much more.
The big data dead valley dilemma and much more.
 
Appengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startupsAppengine vs Amazon; pros & cons for startups
Appengine vs Amazon; pros & cons for startups
 
No BI without Machine Learning
No BI without Machine LearningNo BI without Machine Learning
No BI without Machine Learning
 
Java Empowered by Jython
Java Empowered by JythonJava Empowered by Jython
Java Empowered by Jython
 
easy_install digipy & mlboost
easy_install digipy & mlboosteasy_install digipy & mlboost
easy_install digipy & mlboost
 
Machine Learning empowered by Python April2009
Machine Learning empowered by Python April2009Machine Learning empowered by Python April2009
Machine Learning empowered by Python April2009
 
Intro to Machine Learning Enpowered by Python (Montreal Python)
Intro to Machine Learning Enpowered by Python (Montreal Python)Intro to Machine Learning Enpowered by Python (Montreal Python)
Intro to Machine Learning Enpowered by Python (Montreal Python)
 
Master Defense Slides (translated)
Master Defense Slides (translated)Master Defense Slides (translated)
Master Defense Slides (translated)
 
Soutenance 17 Avril 2003
Soutenance 17 Avril 2003Soutenance 17 Avril 2003
Soutenance 17 Avril 2003
 

How we Leverage Cloud, Big Data, Social Data at QMining (Salon BI nov 6 2012)

  • 1. How we leverage the cloud, big- data, social data to do BI? Salon BI 6 November 2012 francis@qmining.com Founder & CEO
  • 2. Hidden Agenda ● Story ● QMining ● BI landscape - Issues ● How QMining is leveraging: ○ Open source ○ Cloud ○ Hadoop for Big data ○ Social data ● Innovation ○ QMiner ○ QMarketing
  • 4. Products and Consulting Big Data Knowledge Action
  • 5. Big Data Predictive Analysis Patterns/clusters Any data including Knowledge web traffic, social data, e-commerce transactions Action
  • 6. BI Landscape - Issues ● Frustrating ○ not simple/not fast/costly ● Increasing data & sources ● User want more power (bypass IT)
  • 7. QMining ● Simple/fast/cost effective ● Scalable ● More power to the users (bypass IT)
  • 9. Examples Context Enterprise QMining Big Player (content) 18 mois = failure 3 months = success online marketing 180+ days = not 11 days = success done online KPI 30+ days 2 days (bots) investigations IBM SPSS 15 days 1 day ... ...
  • 11. Applications Online Marketing ex: Marketing Management Platform IT Application Management payment ex: performance monitoring & QMining E-commerce ex: merchant retention troubleshooting IT Service Management ex: business process automation
  • 12. How?
  • 13. Methodology Principles: ● Open source ● Minimize Abstraction ● Cloud Tools: ● Python ● Hadoop (for Big Data)
  • 14. Open Source Leverage collaboration and openness to move faster -71% of clients don't use open source
  • 15. All abstractions leak Abstract -> Procrastinate! http://www.aleax.it/pycon_abst.pdf (Alex Martelli : "Abstraction as a Leverage" )
  • 16. Minimize A Tower of Abstraction Simplify & lower the layer of abstraction Examples: ● Work on file not BD if possible ● Low level linux command lines (cut, grep, sed etc.) ● High level languages : python ● Minimize # of languages; ex: Hadoop streaming (map/reduce=python) ● Prototype on a laptop (atom N450)
  • 17. Cloud Leverage IAAS & PAAS infrastructure to build new SaaS
  • 18. Cloud - PaaS vs IaaS Pros ● framework ● less limits ● deployment ● Scaling ● multi-tenant ● Pricing ● https ● log & data visualisation General Score +20 -20 Recommendations: Protot Productio ype n The best of both world might be heroku a Cloud application platform but we haven't tried it montreal python slides on a comparison (appengine vs amazon, pris & cons for a startup)
  • 19. QMining in Cloud Principles: ● Easy deployment of environments ● Scalable by design ● Multi-tenant by design (don't manage multiple versions/instances) ● REST API by design
  • 20. the power of matlab for prod What How power of PaaS (Platform as a Service) Web framework example coradiant : 100X speedup
  • 21. Big Data Leverage cloud to process lot of data (hadoop) We use amazon EMR (hadoop, EC2 & S3)
  • 23. Why enterprises fail at innovation? (blog) 1. Salon BI 2011: Philippe Nieuwbourg M&A 2. Boston CXO Forum 24 October : Best Practice on Global Innovation (IBM, EMC, P&G, Intuit) Exploit vs Explore 3. Brad Feld (Managing Director at Foundry Group) Startup community 29 October Notman house MTL Hierarchy vs network
  • 24. QMiner 1. Simplify real user management 2. Simplify business view management 3. Simplify Persona management
  • 25. QMiner Simplify real user management 10% of your online users are experiencing slowness ● 100% are located in China ● 75% are using mobile device ● 90% are using iphone http://qmining.qminer.com
  • 26. QMiner & Social Graph Leverage the social graph apis Facebook Social Graph: http://developers.facebook.com/docs/reference/api/ QMiner Example: http://fr.qmining.com/facebook/
  • 27. QMiner Simplify business view management Your are losing $10K/hour: ● 100% are Male ● 75% are using mobile device ● 90% are using iphone http://qmining.qminer.com/api/clients/qmining/apps/fbtest/ruleset/
  • 28. QMiner Simplify persona management None buying customers are: 93% male 67% single 87% high educated Persona fit = 25%
  • 30. Investing in online marketing and not getting the expected ROI?
  • 31. You can't find a global solution in this fragmented market?
  • 32. QMarketing - Marketing Management Optimize your online Marketing investments
  • 33. Hadoop for Channel Optimization
  • 34. Non sales Patterns with social data
  • 35. Summary ● Story ● QMining ● BI landscape - Issues (not simple, slow, costly) ● How QMining is leveraging: ○ Open source with linux, python, github ○ Cloud with amazon, google appengine ○ Big data with hadoop ○ Social data with facebook api ● Innovation ○ QMiner Simplify real user, business view & persona management ○ QMarketing simplify online marketing management