SlideShare uma empresa Scribd logo
1 de 36
Baixar para ler offline
Using Data to
         Drive User
        Experiences




        Jeff Patti         jpatti@monetate.com
        Ram Parthasarathy ram@monetate.com
Monday, February 25, 13
Personalization
              On the web, one size doesn't fit all.
              • Demographics
              • Behavior
              • Device (laptop vs tablet vs phone)
              Algorithmic Personalization
              •More relevance, reduced noise to useful
              information ratio. (eg: your facebook feed)



Monday, February 25, 13
Amazon Homepage (Jeff)




Monday, February 25, 13
Amazon Homepage (Ram)




Monday, February 25, 13
Not personal anymore!




Monday, February 25, 13
User Data is Precious
           Every two days, we create as much information as we
           did up to 2003.
           – Eric Schmidt, then-CEO of Google

           Facebook ingests 500+terabytes of new data into their
           databases every day.

           “If you aren’t taking advantage of big data, then you
           don’t have big data, you have just a pile of data,”
           - Jay Parikh, VP of infrastructure at Facebook


Monday, February 25, 13
... And Scary Too!




Monday, February 25, 13
Relevant Search Results - Good!




                             IP spoofed with a proxy



Monday, February 25, 13
Recos based on purchases - Good!




Monday, February 25, 13
Info based on user action - Good!




Monday, February 25, 13
Excessive Ad Retargeting - Bad!




Monday, February 25, 13
Identity Theft, Selling data - Ugly!




Monday, February 25, 13
We provide the data!

                                      • Personal Info - Name,
                                      DOB, family, education
                                      • Interests and Hobbies
                                      • Physical places visited
                                      • Websites visited
                                      • Contacts on the phone




Monday, February 25, 13
They "trust me"
     Zuck: Yeah so if you ever need info about anyone at
     Harvard

     Zuck: Just ask.

     Zuck: I have over 4,000 emails, pictures, addresses, SNS

     [Redacted Friend's Name]: What? How'd you manage that
     one?

     Zuck: People just submitted it.

     Zuck: I don't know why.

     Zuck: They "trust me"

     Zuck: Dumb f****.
Monday, February 25, 13
Personalized Results - Good/Bad?




Monday, February 25, 13
DuckDuckGo




Monday, February 25, 13
A/B Testing




                           A - original version of website
                          B - modified version of website

                   Show each version to different segments of
                   website visitors and track metrics measuring
                              performance of visitors

                                                         source: visualwebsiteoptimizwer

Monday, February 25, 13
Obama Campaign A/B Testing
      http://kylerush.net/blog/optimization-at-the-obama-
      campaign-ab-testing/

      Both Obama campaigns made extensive use of A/B
      testing to optimize their websites




Monday, February 25, 13
Obama Campaign A/B Testing




Monday, February 25, 13
Imitation is the sincerest form of flattery




Monday, February 25, 13
Monday, February 25, 13
Monday, February 25, 13
Why A/B Testing




              Schrödinger's cat - The cat is either alive or
              dead, not both alive and dead.

Monday, February 25, 13
Which Version Won?




                                         3.95%



Monday, February 25, 13
Which Version Won?




                                      5.78%




Monday, February 25, 13
Which Version Won?




                                               Source: visualwebsiteoptimizer.com

Monday, February 25, 13
How Monetate Works
         change anything, anywhere and
             at anytime on a website




                 Default Page   Campaign Delivered   Optimized Version


Monday, February 25, 13
a smarter testing approach
                                                    Optimized
                                                 version delivered
            Your Default
               Page
                                           Fast
                                      (Asynchronous)
                                          change


                            Campaign rules
                               applied



                           ExpressTag fires on
                               page load




Monday, February 25, 13
Fast and Easy UI
                           A Sentence based interface




Monday, February 25, 13
20%




                                        20%
                                of peak ecommerce in holiday 2011 & 2012



Monday, February 25, 13
Monetate Stack
       • AWS - hundreds of machines spread across multiple
            regions
       • Sharded and Replicated MySQL
       • Elastic Map Reduce using mrjob, Mahout, and Hive
       • Python on the backend
       • Google closure tools in the UI and front-end
       • Git and Github for version control.


Monday, February 25, 13
Data Flows
       • Real time system for executing A/B tests
         • Uses MySQL, a relational database
         • Good for many small operations quickly
       • Post hoc analysis of experiments
         • Uses Elastic Map Reduce using mrjob, Mahout, and
                 Hive
            • Good for batch processing - large jobs with high
                 latency

Monday, February 25, 13
Map/Reduce Example - Word Count




Monday, February 25, 13
Role of Computer Science
           • Design - User Interface Design and UX
           • Speed and performance optimization
           • Cloud computing - scale and security
           • Algorithm design and development
           • Building data models and prediction systems
           • High performance web engineering




Monday, February 25, 13
We are hiring!

                          Interns and Software Engineers

                               monetate.com/jobs




Monday, February 25, 13
Thank You!

                            Questions?




Monday, February 25, 13

Mais conteúdo relacionado

Semelhante a Using Data to Drive User Experiences

By Pixel and Pedal: Saving the Planet Through B Power
By Pixel and Pedal: Saving the Planet Through B PowerBy Pixel and Pedal: Saving the Planet Through B Power
By Pixel and Pedal: Saving the Planet Through B PowerTim Frick
 
BigML Webcast: September 25, 2013
BigML Webcast:  September 25, 2013BigML Webcast:  September 25, 2013
BigML Webcast: September 25, 2013BigML, Inc
 
Bartek Kiszala Startup Stage#1 'Growth Hacking'
Bartek Kiszala Startup Stage#1 'Growth Hacking'Bartek Kiszala Startup Stage#1 'Growth Hacking'
Bartek Kiszala Startup Stage#1 'Growth Hacking'Startup Stage
 
Lean Validation Tactics
Lean Validation TacticsLean Validation Tactics
Lean Validation TacticsDanny Boice
 
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to Know
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to KnowBecoming A BI Rockstar: Tips and Tricks Every Analyst Needs to Know
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to KnowIronEdge Group
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19Lean Analytics
 
The Mobile Opportunity March '13 - Creative Licence Digital
The Mobile Opportunity March '13 - Creative Licence DigitalThe Mobile Opportunity March '13 - Creative Licence Digital
The Mobile Opportunity March '13 - Creative Licence DigitalCreativeLicenceDigital
 
WordPress for Beginners - YES Montreal
WordPress for Beginners - YES MontrealWordPress for Beginners - YES Montreal
WordPress for Beginners - YES MontrealKathryn Presner
 
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherStructure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherGigaom
 
HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT
  HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT  HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT
HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUTKyle Goodfriend
 
Viking methodology
Viking methodologyViking methodology
Viking methodologyKarl Groves
 
Mobile API Design Techniques
Mobile API Design TechniquesMobile API Design Techniques
Mobile API Design TechniquesTrieu Nguyen
 
Everything Counts / #OSCON 2013
Everything Counts / #OSCON 2013Everything Counts / #OSCON 2013
Everything Counts / #OSCON 2013Mari Huertas
 
WebSphere User Group UK: Larger Applications with Worklight
WebSphere User Group UK: Larger Applications with WorklightWebSphere User Group UK: Larger Applications with Worklight
WebSphere User Group UK: Larger Applications with WorklightAndrew Ferrier
 
Message Oriented Architecture - Gr8conf US 2013
Message Oriented Architecture - Gr8conf US 2013Message Oriented Architecture - Gr8conf US 2013
Message Oriented Architecture - Gr8conf US 2013Steve Pember
 
Using data to guide product development
Using data to guide product developmentUsing data to guide product development
Using data to guide product developmentMat Clayton
 
The Rise of BaaS A Utopia for Client-Side Developers
The Rise of BaaS A Utopia for Client-Side DevelopersThe Rise of BaaS A Utopia for Client-Side Developers
The Rise of BaaS A Utopia for Client-Side DevelopersMarc Manthey
 

Semelhante a Using Data to Drive User Experiences (20)

By Pixel and Pedal: Saving the Planet Through B Power
By Pixel and Pedal: Saving the Planet Through B PowerBy Pixel and Pedal: Saving the Planet Through B Power
By Pixel and Pedal: Saving the Planet Through B Power
 
BigML Webcast: September 25, 2013
BigML Webcast:  September 25, 2013BigML Webcast:  September 25, 2013
BigML Webcast: September 25, 2013
 
Bartek Kiszala Startup Stage#1 'Growth Hacking'
Bartek Kiszala Startup Stage#1 'Growth Hacking'Bartek Kiszala Startup Stage#1 'Growth Hacking'
Bartek Kiszala Startup Stage#1 'Growth Hacking'
 
Lean Validation Tactics
Lean Validation TacticsLean Validation Tactics
Lean Validation Tactics
 
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to Know
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to KnowBecoming A BI Rockstar: Tips and Tricks Every Analyst Needs to Know
Becoming A BI Rockstar: Tips and Tricks Every Analyst Needs to Know
 
UX, UI, WTF
UX, UI, WTFUX, UI, WTF
UX, UI, WTF
 
Startup metrics toronto March 19
Startup metrics toronto March 19Startup metrics toronto March 19
Startup metrics toronto March 19
 
The Mobile Opportunity March '13 - Creative Licence Digital
The Mobile Opportunity March '13 - Creative Licence DigitalThe Mobile Opportunity March '13 - Creative Licence Digital
The Mobile Opportunity March '13 - Creative Licence Digital
 
WordPress for Beginners - YES Montreal
WordPress for Beginners - YES MontrealWordPress for Beginners - YES Montreal
WordPress for Beginners - YES Montreal
 
ModUX keynote
ModUX keynoteModUX keynote
ModUX keynote
 
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherStructure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
 
HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT
  HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT  HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT
HIDDEN GEMS IN PBCS—THE BENEFITS THEY DON’T TELL YOU ABOUT
 
Faster mobile sites
Faster mobile sitesFaster mobile sites
Faster mobile sites
 
Viking methodology
Viking methodologyViking methodology
Viking methodology
 
Mobile API Design Techniques
Mobile API Design TechniquesMobile API Design Techniques
Mobile API Design Techniques
 
Everything Counts / #OSCON 2013
Everything Counts / #OSCON 2013Everything Counts / #OSCON 2013
Everything Counts / #OSCON 2013
 
WebSphere User Group UK: Larger Applications with Worklight
WebSphere User Group UK: Larger Applications with WorklightWebSphere User Group UK: Larger Applications with Worklight
WebSphere User Group UK: Larger Applications with Worklight
 
Message Oriented Architecture - Gr8conf US 2013
Message Oriented Architecture - Gr8conf US 2013Message Oriented Architecture - Gr8conf US 2013
Message Oriented Architecture - Gr8conf US 2013
 
Using data to guide product development
Using data to guide product developmentUsing data to guide product development
Using data to guide product development
 
The Rise of BaaS A Utopia for Client-Side Developers
The Rise of BaaS A Utopia for Client-Side DevelopersThe Rise of BaaS A Utopia for Client-Side Developers
The Rise of BaaS A Utopia for Client-Side Developers
 

Using Data to Drive User Experiences

  • 1. Using Data to Drive User Experiences Jeff Patti jpatti@monetate.com Ram Parthasarathy ram@monetate.com Monday, February 25, 13
  • 2. Personalization On the web, one size doesn't fit all. • Demographics • Behavior • Device (laptop vs tablet vs phone) Algorithmic Personalization •More relevance, reduced noise to useful information ratio. (eg: your facebook feed) Monday, February 25, 13
  • 6. User Data is Precious Every two days, we create as much information as we did up to 2003. – Eric Schmidt, then-CEO of Google Facebook ingests 500+terabytes of new data into their databases every day. “If you aren’t taking advantage of big data, then you don’t have big data, you have just a pile of data,” - Jay Parikh, VP of infrastructure at Facebook Monday, February 25, 13
  • 7. ... And Scary Too! Monday, February 25, 13
  • 8. Relevant Search Results - Good! IP spoofed with a proxy Monday, February 25, 13
  • 9. Recos based on purchases - Good! Monday, February 25, 13
  • 10. Info based on user action - Good! Monday, February 25, 13
  • 11. Excessive Ad Retargeting - Bad! Monday, February 25, 13
  • 12. Identity Theft, Selling data - Ugly! Monday, February 25, 13
  • 13. We provide the data! • Personal Info - Name, DOB, family, education • Interests and Hobbies • Physical places visited • Websites visited • Contacts on the phone Monday, February 25, 13
  • 14. They "trust me" Zuck: Yeah so if you ever need info about anyone at Harvard Zuck: Just ask. Zuck: I have over 4,000 emails, pictures, addresses, SNS [Redacted Friend's Name]: What? How'd you manage that one? Zuck: People just submitted it. Zuck: I don't know why. Zuck: They "trust me" Zuck: Dumb f****. Monday, February 25, 13
  • 15. Personalized Results - Good/Bad? Monday, February 25, 13
  • 17. A/B Testing A - original version of website B - modified version of website Show each version to different segments of website visitors and track metrics measuring performance of visitors source: visualwebsiteoptimizwer Monday, February 25, 13
  • 18. Obama Campaign A/B Testing http://kylerush.net/blog/optimization-at-the-obama- campaign-ab-testing/ Both Obama campaigns made extensive use of A/B testing to optimize their websites Monday, February 25, 13
  • 19. Obama Campaign A/B Testing Monday, February 25, 13
  • 20. Imitation is the sincerest form of flattery Monday, February 25, 13
  • 23. Why A/B Testing Schrödinger's cat - The cat is either alive or dead, not both alive and dead. Monday, February 25, 13
  • 24. Which Version Won? 3.95% Monday, February 25, 13
  • 25. Which Version Won? 5.78% Monday, February 25, 13
  • 26. Which Version Won? Source: visualwebsiteoptimizer.com Monday, February 25, 13
  • 27. How Monetate Works change anything, anywhere and at anytime on a website Default Page Campaign Delivered Optimized Version Monday, February 25, 13
  • 28. a smarter testing approach Optimized version delivered Your Default Page Fast (Asynchronous) change Campaign rules applied ExpressTag fires on page load Monday, February 25, 13
  • 29. Fast and Easy UI A Sentence based interface Monday, February 25, 13
  • 30. 20% 20% of peak ecommerce in holiday 2011 & 2012 Monday, February 25, 13
  • 31. Monetate Stack • AWS - hundreds of machines spread across multiple regions • Sharded and Replicated MySQL • Elastic Map Reduce using mrjob, Mahout, and Hive • Python on the backend • Google closure tools in the UI and front-end • Git and Github for version control. Monday, February 25, 13
  • 32. Data Flows • Real time system for executing A/B tests • Uses MySQL, a relational database • Good for many small operations quickly • Post hoc analysis of experiments • Uses Elastic Map Reduce using mrjob, Mahout, and Hive • Good for batch processing - large jobs with high latency Monday, February 25, 13
  • 33. Map/Reduce Example - Word Count Monday, February 25, 13
  • 34. Role of Computer Science • Design - User Interface Design and UX • Speed and performance optimization • Cloud computing - scale and security • Algorithm design and development • Building data models and prediction systems • High performance web engineering Monday, February 25, 13
  • 35. We are hiring! Interns and Software Engineers monetate.com/jobs Monday, February 25, 13
  • 36. Thank You! Questions? Monday, February 25, 13