SlideShare uma empresa Scribd logo
1 de 117
PUZZLING
                      SPEAKER: Jeff Jonas
                               Chief Scientist
                               Entity Analytics
                               IBM

Tuesday, November 27, 12
Puzzling
                              How Context Accumulates



                           Jeff Jonas, IBM Distinguished Engineer
                            Chief Scientist, IBM Entity Analytics

                                  Email: jeffjonas@us.ibm.com
                                Blog: www.jeffjonas.typepad.com
                           Twitter: http://www.twitter.com/jeffjonas


Tuesday, November 27, 12
FIRE




Tuesday, November 27, 12
FIRE




Tuesday, November 27, 12
You can’t squeeze knowledge out of a pixel.




Tuesday, November 27, 12
State of the Union: “Pixel Analytics”




        Observation                            Consumer
          Space                             (An analyst, a system,
                                            the sensor itself, etc.)

Tuesday, November 27, 12
State of the Union: “Pixel Analytics”




        Observation                            Consumer
          Space                             (An analyst, a system,
                                            the sensor itself, etc.)

Tuesday, November 27, 12
State of the Union: “Pixel Analytics”
                           Red Puzzle Piece Analytics




        Observation                                        Consumer
          Space                                         (An analyst, a system,
                                                        the sensor itself, etc.)

Tuesday, November 27, 12
State of the Union: “Pixel Analytics”
                           Red Puzzle Piece Analytics



                            Green Puzzle Piece Analytics




        Observation                                           Consumer
          Space                                            (An analyst, a system,
                                                           the sensor itself, etc.)

Tuesday, November 27, 12
State of the Union: “Pixel Analytics”
                           Red Puzzle Piece Analytics



                            Green Puzzle Piece Analytics



                                Blue Puzzle Piece Analytics




        Observation                                              Consumer
          Space                                               (An analyst, a system,
                                                              the sensor itself, etc.)

Tuesday, November 27, 12
Without context … quality predictions are hard to come by.




Tuesday, November 27, 12
Context      definition



              Better understanding something by
              taking into account the things around it.



Tuesday, November 27, 12
First … The Data Must Find the Data




         Observation                         Consumer
           Space                          (An analyst, a system,
                                          the sensor itself, etc.)


Tuesday, November 27, 12
First … The Data Must Find the Data


                           Context Accumulation




         Observation                              Persistent      Consumer
           Space                                   Context     (An analyst, a system,
                                                               the sensor itself, etc.)


Tuesday, November 27, 12
First … The Data Must Find the Data

                                                               Relevance Detection
                           Context Accumulation




         Observation                              Persistent                         Consumer
           Space                                   Context                      (An analyst, a system,
                                                                                the sensor itself, etc.)


Tuesday, November 27, 12
Big Data




                           Pile of ____   In Context

Tuesday, November 27, 12
Big Data [in context]. New Physics.




Tuesday, November 27, 12
Big Data [in context]. New Physics.

                   §More data: better the predictions
                     – Lower false positives
                     – Lower false negatives

                   §More data: bad data good
                     – Suddenly glad your data is not perfect




Tuesday, November 27, 12
Big Data [in context]. New Physics.

                   §More data: better the predictions
                     – Lower false positives
                     – Lower false negatives

                   §More data: bad data good
                     – Suddenly glad your data is not perfect


                   §More data: less compute

Tuesday, November 27, 12
Puzzling



                Vegas
                Artwork provided by Hadley
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                All Rights Reserved
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                All Rights Reserved
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                                                Cushman Hayes
                                                © 2009 Ravensburger USA, Inc.
                All Rights Reserved
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                                                     66%
                                                Cushman Hayes
                                                © 2009 Ravensburger USA, Inc.
                All Rights Reserved
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                                                                                Down Home Music
                                                                                © Kay Lamb Shannon, Artist
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                                                     66%
                                                Cushman Hayes
                                                © 2009 Ravensburger USA, Inc.
                                                                                Licensed by Cypress Fine Art
                                                                                Licensing
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                                                                                  150 pieces
                                                                                Down Home Music
                                                                                © Kay Lamb Shannon, Artist
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                                                     66%
                                                Cushman Hayes
                                                © 2009 Ravensburger USA, Inc.
                                                                                     50%
                                                                                Licensed by Cypress Fine Art
                                                                                Licensing
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                                                                                  150 pieces
                                                                                Down Home Music
                                                                                © Kay Lamb Shannon, Artist
                                                                                                               Cottage Garden
                                                                                                               © 2010 Royce B. McClure, Artist
                      90%
                House Licensing, Minneapolis
                © 2011 Giesla Hoelscher
                                                     66%
                                                Cushman Hayes
                                                © 2009 Ravensburger USA, Inc.
                                                                                     50%
                                                                                Licensed by Cypress Fine Art
                                                                                Licensing
                                                                                                               All Rights Reserved
                                                                                                               © 2010 Ravensburger USA, Inc.
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                                                                                  150 pieces
                                                                                Down Home Music
                                                                                © Kay Lamb Shannon, Artist           6 pieces
                                                                                                               Cottage Garden
                                                                                                               © 2010 Royce B. McClure, Artist
                      90%                            66%                             50%                                2%
                House Licensing, Minneapolis    Cushman Hayes                   Licensed by Cypress Fine Art   All Rights Reserved
                © 2011 Giesla Hoelscher         © 2009 Ravensburger USA, Inc.   Licensing                      © 2010 Ravensburger USA, Inc.
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Puzzling



                   270 pieces
                Vegas
                Artwork provided by Hadley
                                                  200 pieces
                                                Neuschwanstein Beauty
                                                © 2009 Photo Copyright Robert
                                                                                  150 pieces
                                                                                Down Home Music
                                                                                © Kay Lamb Shannon, Artist           6 pieces
                                                                                                               Cottage Garden
                                                                                                               © 2010 Royce B. McClure, Artist
                      90%                            66%                             50%                                2%
                House Licensing, Minneapolis    Cushman Hayes                   Licensed by Cypress Fine Art   All Rights Reserved
                © 2011 Giesla Hoelscher         © 2009 Ravensburger USA, Inc.   Licensing                      © 2010 Ravensburger USA, Inc.
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




                                                30 pieces
                                                10%
                                                (duplicates)


Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Incremental Context – Incremental Discovery




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      6:40pm               START




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      6:40pm               START

      22min                “Hey, this one is a duplicate!”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      6:40pm               START

      22min                “Hey, this one is a duplicate!”

      35min                “I think some pieces are missing.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      6:40pm               START

      22min                “Hey, this one is a duplicate!”

      35min                “I think some pieces are missing.”

      37min                “Looks like a bunch of hillbillies on a porch.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      6:40pm               START

      22min                “Hey, this one is a duplicate!”

      35min                “I think some pieces are missing.”

      37min                “Looks like a bunch of hillbillies on a porch.”

      44min                “Hillbillies, playing guitars, sitting on a porch,
                           near a barber sign … and a banjo!”


Tuesday, November 27, 12
Tuesday, November 27, 12
150 pieces
                                              50%

          Down Home Music
          © Kay Lamb Shannon, Artist.
          Licensed by Cypress Fine Art Licensing
          © 2011 Ravensburger USA Inc.




Tuesday, November 27, 12
Incremental Context – Incremental Discovery




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      47min                “We should take the sky and grass off the table.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      47min                “We should take the sky and grass off the table.”

      2hr                  “Let’s switch sides, and see if we can make sense
                           of this from different perspectives.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      47min                “We should take the sky and grass off the table.”

      2hr                  “Let’s switch sides, and see if we can make sense
                           of this from different perspectives.”

      2hr10m               “Wait, there are three … no, four puzzles.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      47min                “We should take the sky and grass off the table.”

      2hr                  “Let’s switch sides, and see if we can make sense
                           of this from different perspectives.”

      2hr10m               “Wait, there are three … no, four puzzles.”

      2hr17m               “We need a bigger table.”




Tuesday, November 27, 12
Incremental Context – Incremental Discovery


      47min                “We should take the sky and grass off the table.”

      2hr                  “Let’s switch sides, and see if we can make sense
                           of this from different perspectives.”

      2hr10m               “Wait, there are three … no, four puzzles.”

      2hr17m               “We need a bigger table.”

      2hr18m               “I think you threw in a few random pieces.”


Tuesday, November 27, 12
Tuesday, November 27, 12
How Context Accumulates




Tuesday, November 27, 12
How Context Accumulates
      § With each new observation one of three assertions are made:

            1) Un-associated; 2) Placed near like neighbors; or 3) Connected




Tuesday, November 27, 12
How Context Accumulates
      § With each new observation one of three assertions are made:

            1) Un-associated; 2) Placed near like neighbors; or 3) Connected

      § Must favor the false negative




Tuesday, November 27, 12
How Context Accumulates
      § With each new observation one of three assertions are made:

            1) Un-associated; 2) Placed near like neighbors; or 3) Connected

      § Must favor the false negative

      § New observations sometimes reverse earlier assertions




Tuesday, November 27, 12
How Context Accumulates
      § With each new observation one of three assertions are made:

            1) Un-associated; 2) Placed near like neighbors; or 3) Connected

      § Must favor the false negative

      § New observations sometimes reverse earlier assertions

      § As the working space expands, computational effort increases




Tuesday, November 27, 12
How Context Accumulates
      § With each new observation one of three assertions are made:

            1) Un-associated; 2) Placed near like neighbors; or 3) Connected

      § Must favor the false negative

      § New observations sometimes reverse earlier assertions

      § As the working space expands, computational effort increases

      § Given sufficient observations, there can come a tipping point.
         Thereafter, confidence improves while computational effort
         decreases!

Tuesday, November 27, 12
Puzzling Experiment #2


                           “Testing the Fast Last Puzzle Piece”




Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Sensemaking on Streams

      § Each person gets one piece per round, no collaborating


      § Only work the piece
            – Figure out where it goes
            – If you stumble upon something else worth fixing, fix it
            – When there is no more to do on that piece, stop and say you are done


      § If you have new insight, tell me


      § Each assembler is timed for each piece

Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Deep Reflection and Consolidation

      § Every 10 rounds (40 pieces) you can re-consider what is already known and
        collaborate while doing this


      § Spend as much time as needed, until not much more can be accomplished


      § Puzzle chunks are counted before and after




Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Noteworthy Events




Tuesday, November 27, 12
Noteworthy Events

      § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!”




Tuesday, November 27, 12
Noteworthy Events

      § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!”

      § @ 4% (12 pieces) the first two pieces connect




Tuesday, November 27, 12
Noteworthy Events

      § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!”

      § @ 4% (12 pieces) the first two pieces connect

      § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds

            – Why? “Never seen anything like it.”

      § @ 48% (144 pieces) new insight:

            – “Big welcome Las Vegas sign with everything from the strip around it.”




Tuesday, November 27, 12
Noteworthy Events

      § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!”

      § @ 4% (12 pieces) the first two pieces connect

      § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds

            – Why? “Never seen anything like it.”

      § @ 48% (144 pieces) new insight:

            – “Big welcome Las Vegas sign with everything from the strip around it.”

      § @ 65% (196 pieces) the first false positive is detected and corrected




Tuesday, November 27, 12
Noteworthy Events

      § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!”

      § @ 4% (12 pieces) the first two pieces connect

      § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds

            – Why? “Never seen anything like it.”

      § @ 48% (144 pieces) new insight:

            – “Big welcome Las Vegas sign with everything from the strip around it.”

      § @ 65% (196 pieces) the first false positive is detected and corrected

      § @ 75% (224 pieces) new insight: “It is getting easier.”


Tuesday, November 27, 12
Tuesday, November 27, 12
Compute Effort as Observation Space Grows




Tuesday, November 27, 12
Compute Effort as Observation Space Grows



                                     57->55

                                              87->61


                           36->34                               54->29

                                    83->72
                                                       82->54



                                                                         55->1




Tuesday, November 27, 12
Compute Effort as Observation Space Grows



                                     57->55

                                              87->61


                           36->34                               54->29

                                    83->72
                                                       82->54



                                                                         55->1




Tuesday, November 27, 12
More Data, Less Compute




Tuesday, November 27, 12
Lessons Learned




Tuesday, November 27, 12
Lessons Learned

               The last piece was almost as fast as the first.




Tuesday, November 27, 12
Lessons Learned

               The last piece was almost as fast as the first.


          Deep reflection (batch-based pattern discovery)
                          was significantly
                more important than I had thought.



Tuesday, November 27, 12
Puzzling Experiment #3


                                  “Adults”



Tuesday, November 27, 12
SIBOS Conference 2011




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams
                           § 10 puzzles, 10 small tables




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams
                           § 10 puzzles, 10 small tables
                           § Duplicate and missing pieces




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams
                           § 10 puzzles, 10 small tables
                           § Duplicate and missing pieces

                           Lessons:




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams
                           § 10 puzzles, 10 small tables
                           § Duplicate and missing pieces

                           Lessons:

                           1. They learned federated search bites.




Tuesday, November 27, 12
SIBOS Conference 2011

                           § 100 executives, 10 teams
                           § 10 puzzles, 10 small tables
                           § Duplicate and missing pieces

                           Lessons:

                           1. They learned federated search bites.

                           2. I watched as an early bias misdirected
                            their attention … but then over time
                            new observations corrected this bias.


Tuesday, November 27, 12
Puzzling Experiment #4


                            “Not an Experiment”



Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Tuesday, November 27, 12
Puzzling Project #4: Commentary




Tuesday, November 27, 12
Puzzling Project #4: Commentary

      § Despite having only 100 pieces and eight collaborating eyeballs:
             – We began to suspect there were missing and random pieces
             – We had an alarming number of false positives1
             – It took significantly more effort/time than expected




        1
            The primary source being one overly intoxicated pipeline.


Tuesday, November 27, 12
Puzzling Project #4: Commentary

      § Despite having only 100 pieces and eight collaborating eyeballs:
             – We began to suspect there were missing and random pieces
             – We had an alarming number of false positives1
             – It took significantly more effort/time than expected


      § Why? Common shapes and that vast purple haze (lots of ambiguity).




        1
            The primary source being one overly intoxicated pipeline.


Tuesday, November 27, 12
Experiment #4: Notes to Self




Tuesday, November 27, 12
Experiment #4: Notes to Self

      § Excessive ambiguity drives computational cost way up




Tuesday, November 27, 12
Experiment #4: Notes to Self

      § Excessive ambiguity drives computational cost way up




Tuesday, November 27, 12
Experiment #4: Notes to Self

      § Excessive ambiguity drives computational cost way up


      § Some drunk people get unreasonably optimistic




Tuesday, November 27, 12
My Recommendations




Tuesday, November 27, 12
My Recommendations

      § Context Accumulation
            – Investments in general purpose information fusion will often yield greater value
              than investments in specialized, single-sensor, algorithms (pixel analytics).




Tuesday, November 27, 12
My Recommendations

      § Context Accumulation
            – Investments in general purpose information fusion will often yield greater value
              than investments in specialized, single-sensor, algorithms (pixel analytics).


      § Real-time Analytics Over Big Data
            – If something can be engineered for real-time, do that. There is a competitive
              advantage when one can respond intelligently while a transaction is still happening.




Tuesday, November 27, 12
My Recommendations

      § Context Accumulation
            – Investments in general purpose information fusion will often yield greater value
              than investments in specialized, single-sensor, algorithms (pixel analytics).


      § Real-time Analytics Over Big Data
            – If something can be engineered for real-time, do that. There is a competitive
              advantage when one can respond intelligently while a transaction is still happening.


      § Deep Reflection
            – Do not underestimate the value of periodic deep reflection (pattern discovery). Do
              this more often. And put this emerging insight to immediate use via feedback loops.

Tuesday, November 27, 12
Related Blog Posts
            Puzzling: How Observations Are Accumulated Into Context
            Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of A Pixel
            Data Finds Data
            Big Data. New Physics.
            General Purpose Sensemaking Systems and Information Colocation
            Data Beats Math


   And Easy to Reach
             Email: jeffjonas@us.ibm.com
             Twitter: http://www.twitter.com/jeffjonas



Tuesday, November 27, 12
Puzzling
                               How Context Accumulates



                           Jeff Jonas, IBM Distinguished Engineer
                            Chief Scientist, IBM Entity Analytics

                                  Email: jeffjonas@us.ibm.com
                                Blog: www.jeffjonas.typepad.com
                           Twitter: http://www.twitter.com/jeffjonas

Tuesday, November 27, 12
Puzzling




                Vegas                           Neuschwanstein Beauty           Down Home Music                Cottage Garden
                Artwork provided by Hadley      © 2009 Photo Copyright Robert   © Kay Lamb Shannon, Artist     © 2010 Royce B. McClure, Artist
                House Licensing, Minneapolis    Cushman Hayes                   Licensed by Cypress Fine Art   All Rights Reserved
                © 2011 Giesla Hoelscher         © 2009 Ravensburger USA, Inc.   Licensing                      © 2010 Ravensburger USA, Inc.
                All Rights Reserved                                             © 2011 Ravensburger USA Inc.
                © 2011 Ravensburger USA, Inc.




Tuesday, November 27, 12
Tuesday, November 27, 12

Mais conteúdo relacionado

Mais de Gigaom

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...Gigaom
 
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit BendovStructure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit BendovGigaom
 
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...Gigaom
 
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA, Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA, Gigaom
 
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherStructure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherGigaom
 
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadStructure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadGigaom
 
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...Gigaom
 
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrathStructure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrathGigaom
 
Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell
Structure Data 2014: IS VIDEO BIG DATA?, Steve RussellStructure Data 2014: IS VIDEO BIG DATA?, Steve Russell
Structure Data 2014: IS VIDEO BIG DATA?, Steve RussellGigaom
 
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan WaiteStructure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan WaiteGigaom
 
How Data is Remaking E-commerce - from Roadmap 2013
How Data is Remaking E-commerce - from Roadmap 2013How Data is Remaking E-commerce - from Roadmap 2013
How Data is Remaking E-commerce - from Roadmap 2013Gigaom
 
25 Favorite Experiences in Tech - from Roadmap 2013
25 Favorite Experiences in Tech - from Roadmap 201325 Favorite Experiences in Tech - from Roadmap 2013
25 Favorite Experiences in Tech - from Roadmap 2013Gigaom
 
How Moore’s Law is Influencing Design - from Roadmap 2013
How Moore’s Law is Influencing Design - from Roadmap 2013How Moore’s Law is Influencing Design - from Roadmap 2013
How Moore’s Law is Influencing Design - from Roadmap 2013Gigaom
 
Building the Visual Language of the Web - from Roadmap 2013
Building the Visual Language of the Web - from Roadmap 2013Building the Visual Language of the Web - from Roadmap 2013
Building the Visual Language of the Web - from Roadmap 2013Gigaom
 
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013Gigaom
 
Gigaom Research Sector RoadMap: Enterprise Mobility Management
Gigaom Research Sector RoadMap: Enterprise Mobility Management Gigaom Research Sector RoadMap: Enterprise Mobility Management
Gigaom Research Sector RoadMap: Enterprise Mobility Management Gigaom
 
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013Gigaom
 
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013Gigaom
 
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 20135 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013Gigaom
 
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013Gigaom
 

Mais de Gigaom (20)

Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
Structure Data 2014: QLIK SPONSOR WORKSHOP: ANALYTICS THE WAY NATURE INTENDED...
 
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit BendovStructure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
Structure Data 2014: FIVE MYTHS ABOUT BIG DATA, Amit Bendov
 
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
Structure Data 2014: AMID BILLIONS OF METRICS, YOUR SOFTWARE IS TRYING TO TEL...
 
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA, Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
Structure Data 2014: SISENSE SPONSOR WORKSHOP: ON BEER, CHIPS AND DATA,
 
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari GesherStructure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
Structure Data 2014: INVERTING 80/20: BEYOND BESPOKE BIG DATA, Ari Gesher
 
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris HaddadStructure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
Structure Data 2014: TRACKING A SOCCER GAME WITH BIG DATA, Chris Haddad
 
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
Structure Data 2014: TECH AGAINST HUMAN TRAFFICKING AND ILLICIT NETWORKS, Jus...
 
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrathStructure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
Structure Data 2014: DATA DRIVEN DESIGN AT FORMULA ONE SPEED, Geoff McGrath
 
Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell
Structure Data 2014: IS VIDEO BIG DATA?, Steve RussellStructure Data 2014: IS VIDEO BIG DATA?, Steve Russell
Structure Data 2014: IS VIDEO BIG DATA?, Steve Russell
 
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan WaiteStructure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
Structure Data 2014: BIG DATA ANALYTICS RE-INVENTED, Ryan Waite
 
How Data is Remaking E-commerce - from Roadmap 2013
How Data is Remaking E-commerce - from Roadmap 2013How Data is Remaking E-commerce - from Roadmap 2013
How Data is Remaking E-commerce - from Roadmap 2013
 
25 Favorite Experiences in Tech - from Roadmap 2013
25 Favorite Experiences in Tech - from Roadmap 201325 Favorite Experiences in Tech - from Roadmap 2013
25 Favorite Experiences in Tech - from Roadmap 2013
 
How Moore’s Law is Influencing Design - from Roadmap 2013
How Moore’s Law is Influencing Design - from Roadmap 2013How Moore’s Law is Influencing Design - from Roadmap 2013
How Moore’s Law is Influencing Design - from Roadmap 2013
 
Building the Visual Language of the Web - from Roadmap 2013
Building the Visual Language of the Web - from Roadmap 2013Building the Visual Language of the Web - from Roadmap 2013
Building the Visual Language of the Web - from Roadmap 2013
 
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013
Your Device Knows What You’re Thinking Before You Do - from Mobilize 2013
 
Gigaom Research Sector RoadMap: Enterprise Mobility Management
Gigaom Research Sector RoadMap: Enterprise Mobility Management Gigaom Research Sector RoadMap: Enterprise Mobility Management
Gigaom Research Sector RoadMap: Enterprise Mobility Management
 
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013
CERN'S GRAND COLLISION EXPERIMENT: CLOUD from Structure:Europe 2013
 
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013
GOOGLE'S INFRASTRUCTURE DNA PACKAGED UP from Structure:Europe 2013
 
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 20135 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013
5 ESSENTIAL TOOLS FOR OPTIMIZING IAAS from Structure:Europe 2013
 
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013
BMW ACCELERATES CLOUD ADOPTION WITH ODCA from Structure:Europe 2013
 

Último

AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024Brian Pichman
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0DanBrown980551
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxKaustubhBhavsar6
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2DianaGray10
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInThousandEyes
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarThousandEyes
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Alkin Tezuysal
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox
 

Último (20)

AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024AI Workshops at Computers In Libraries 2024
AI Workshops at Computers In Libraries 2024
 
SheDev 2024
SheDev 2024SheDev 2024
SheDev 2024
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie WorldTrustArc Webinar - How to Live in a Post Third-Party Cookie World
TrustArc Webinar - How to Live in a Post Third-Party Cookie World
 
LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0LF Energy Webinar - Unveiling OpenEEMeter 4.0
LF Energy Webinar - Unveiling OpenEEMeter 4.0
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
How to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptxHow to become a GDSC Lead GDSC MI AOE.pptx
How to become a GDSC Lead GDSC MI AOE.pptx
 
UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2UiPath Studio Web workshop series - Day 2
UiPath Studio Web workshop series - Day 2
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedInOutage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
Outage Analysis: March 5th/6th 2024 Meta, Comcast, and LinkedIn
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
EMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? WebinarEMEA What is ThousandEyes? Webinar
EMEA What is ThousandEyes? Webinar
 
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
Design and Modeling for MySQL SCALE 21X Pasadena, CA Mar 2024
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through TokenizationStobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
Stobox 4: Revolutionizing Investment in Real-World Assets Through Tokenization
 

PUZZLING- HOW CONTEXT ACCUMULATES from Structure:Data 2012

  • 1. PUZZLING SPEAKER: Jeff Jonas Chief Scientist Entity Analytics IBM Tuesday, November 27, 12
  • 2. Puzzling How Context Accumulates Jeff Jonas, IBM Distinguished Engineer Chief Scientist, IBM Entity Analytics Email: jeffjonas@us.ibm.com Blog: www.jeffjonas.typepad.com Twitter: http://www.twitter.com/jeffjonas Tuesday, November 27, 12
  • 5. You can’t squeeze knowledge out of a pixel. Tuesday, November 27, 12
  • 6. State of the Union: “Pixel Analytics” Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 7. State of the Union: “Pixel Analytics” Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 8. State of the Union: “Pixel Analytics” Red Puzzle Piece Analytics Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 9. State of the Union: “Pixel Analytics” Red Puzzle Piece Analytics Green Puzzle Piece Analytics Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 10. State of the Union: “Pixel Analytics” Red Puzzle Piece Analytics Green Puzzle Piece Analytics Blue Puzzle Piece Analytics Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 11. Without context … quality predictions are hard to come by. Tuesday, November 27, 12
  • 12. Context definition Better understanding something by taking into account the things around it. Tuesday, November 27, 12
  • 13. First … The Data Must Find the Data Observation Consumer Space (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 14. First … The Data Must Find the Data Context Accumulation Observation Persistent Consumer Space Context (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 15. First … The Data Must Find the Data Relevance Detection Context Accumulation Observation Persistent Consumer Space Context (An analyst, a system, the sensor itself, etc.) Tuesday, November 27, 12
  • 16. Big Data Pile of ____ In Context Tuesday, November 27, 12
  • 17. Big Data [in context]. New Physics. Tuesday, November 27, 12
  • 18. Big Data [in context]. New Physics. §More data: better the predictions – Lower false positives – Lower false negatives §More data: bad data good – Suddenly glad your data is not perfect Tuesday, November 27, 12
  • 19. Big Data [in context]. New Physics. §More data: better the predictions – Lower false positives – Lower false negatives §More data: bad data good – Suddenly glad your data is not perfect §More data: less compute Tuesday, November 27, 12
  • 20. Puzzling Vegas Artwork provided by Hadley House Licensing, Minneapolis © 2011 Giesla Hoelscher All Rights Reserved © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 21. Puzzling 270 pieces Vegas Artwork provided by Hadley 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher All Rights Reserved © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 22. Puzzling 270 pieces Vegas Artwork provided by Hadley Neuschwanstein Beauty © 2009 Photo Copyright Robert 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher Cushman Hayes © 2009 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 23. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher 66% Cushman Hayes © 2009 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 24. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert Down Home Music © Kay Lamb Shannon, Artist 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher 66% Cushman Hayes © 2009 Ravensburger USA, Inc. Licensed by Cypress Fine Art Licensing All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 25. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert 150 pieces Down Home Music © Kay Lamb Shannon, Artist 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher 66% Cushman Hayes © 2009 Ravensburger USA, Inc. 50% Licensed by Cypress Fine Art Licensing All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 26. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert 150 pieces Down Home Music © Kay Lamb Shannon, Artist Cottage Garden © 2010 Royce B. McClure, Artist 90% House Licensing, Minneapolis © 2011 Giesla Hoelscher 66% Cushman Hayes © 2009 Ravensburger USA, Inc. 50% Licensed by Cypress Fine Art Licensing All Rights Reserved © 2010 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 27. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert 150 pieces Down Home Music © Kay Lamb Shannon, Artist 6 pieces Cottage Garden © 2010 Royce B. McClure, Artist 90% 66% 50% 2% House Licensing, Minneapolis Cushman Hayes Licensed by Cypress Fine Art All Rights Reserved © 2011 Giesla Hoelscher © 2009 Ravensburger USA, Inc. Licensing © 2010 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12
  • 28. Puzzling 270 pieces Vegas Artwork provided by Hadley 200 pieces Neuschwanstein Beauty © 2009 Photo Copyright Robert 150 pieces Down Home Music © Kay Lamb Shannon, Artist 6 pieces Cottage Garden © 2010 Royce B. McClure, Artist 90% 66% 50% 2% House Licensing, Minneapolis Cushman Hayes Licensed by Cypress Fine Art All Rights Reserved © 2011 Giesla Hoelscher © 2009 Ravensburger USA, Inc. Licensing © 2010 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. 30 pieces 10% (duplicates) Tuesday, November 27, 12
  • 38. Incremental Context – Incremental Discovery Tuesday, November 27, 12
  • 39. Incremental Context – Incremental Discovery 6:40pm START Tuesday, November 27, 12
  • 40. Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” Tuesday, November 27, 12
  • 41. Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” Tuesday, November 27, 12
  • 42. Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” 37min “Looks like a bunch of hillbillies on a porch.” Tuesday, November 27, 12
  • 43. Incremental Context – Incremental Discovery 6:40pm START 22min “Hey, this one is a duplicate!” 35min “I think some pieces are missing.” 37min “Looks like a bunch of hillbillies on a porch.” 44min “Hillbillies, playing guitars, sitting on a porch, near a barber sign … and a banjo!” Tuesday, November 27, 12
  • 45. 150 pieces 50% Down Home Music © Kay Lamb Shannon, Artist. Licensed by Cypress Fine Art Licensing © 2011 Ravensburger USA Inc. Tuesday, November 27, 12
  • 46. Incremental Context – Incremental Discovery Tuesday, November 27, 12
  • 47. Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” Tuesday, November 27, 12
  • 48. Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” Tuesday, November 27, 12
  • 49. Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” Tuesday, November 27, 12
  • 50. Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” 2hr17m “We need a bigger table.” Tuesday, November 27, 12
  • 51. Incremental Context – Incremental Discovery 47min “We should take the sky and grass off the table.” 2hr “Let’s switch sides, and see if we can make sense of this from different perspectives.” 2hr10m “Wait, there are three … no, four puzzles.” 2hr17m “We need a bigger table.” 2hr18m “I think you threw in a few random pieces.” Tuesday, November 27, 12
  • 54. How Context Accumulates § With each new observation one of three assertions are made: 1) Un-associated; 2) Placed near like neighbors; or 3) Connected Tuesday, November 27, 12
  • 55. How Context Accumulates § With each new observation one of three assertions are made: 1) Un-associated; 2) Placed near like neighbors; or 3) Connected § Must favor the false negative Tuesday, November 27, 12
  • 56. How Context Accumulates § With each new observation one of three assertions are made: 1) Un-associated; 2) Placed near like neighbors; or 3) Connected § Must favor the false negative § New observations sometimes reverse earlier assertions Tuesday, November 27, 12
  • 57. How Context Accumulates § With each new observation one of three assertions are made: 1) Un-associated; 2) Placed near like neighbors; or 3) Connected § Must favor the false negative § New observations sometimes reverse earlier assertions § As the working space expands, computational effort increases Tuesday, November 27, 12
  • 58. How Context Accumulates § With each new observation one of three assertions are made: 1) Un-associated; 2) Placed near like neighbors; or 3) Connected § Must favor the false negative § New observations sometimes reverse earlier assertions § As the working space expands, computational effort increases § Given sufficient observations, there can come a tipping point. Thereafter, confidence improves while computational effort decreases! Tuesday, November 27, 12
  • 59. Puzzling Experiment #2 “Testing the Fast Last Puzzle Piece” Tuesday, November 27, 12
  • 62. Sensemaking on Streams § Each person gets one piece per round, no collaborating § Only work the piece – Figure out where it goes – If you stumble upon something else worth fixing, fix it – When there is no more to do on that piece, stop and say you are done § If you have new insight, tell me § Each assembler is timed for each piece Tuesday, November 27, 12
  • 65. Deep Reflection and Consolidation § Every 10 rounds (40 pieces) you can re-consider what is already known and collaborate while doing this § Spend as much time as needed, until not much more can be accomplished § Puzzle chunks are counted before and after Tuesday, November 27, 12
  • 71. Noteworthy Events § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!” Tuesday, November 27, 12
  • 72. Noteworthy Events § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!” § @ 4% (12 pieces) the first two pieces connect Tuesday, November 27, 12
  • 73. Noteworthy Events § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!” § @ 4% (12 pieces) the first two pieces connect § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds – Why? “Never seen anything like it.” § @ 48% (144 pieces) new insight: – “Big welcome Las Vegas sign with everything from the strip around it.” Tuesday, November 27, 12
  • 74. Noteworthy Events § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!” § @ 4% (12 pieces) the first two pieces connect § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds – Why? “Never seen anything like it.” § @ 48% (144 pieces) new insight: – “Big welcome Las Vegas sign with everything from the strip around it.” § @ 65% (196 pieces) the first false positive is detected and corrected Tuesday, November 27, 12
  • 75. Noteworthy Events § @ 1.3% (4 pieces) new insight: “It’s Las Vegas and Sahara Hotel!” § @ 4% (12 pieces) the first two pieces connect § @ 37% (112 pieces) a puzzle piece is processed by a “pipeline” in 2.7 seconds – Why? “Never seen anything like it.” § @ 48% (144 pieces) new insight: – “Big welcome Las Vegas sign with everything from the strip around it.” § @ 65% (196 pieces) the first false positive is detected and corrected § @ 75% (224 pieces) new insight: “It is getting easier.” Tuesday, November 27, 12
  • 77. Compute Effort as Observation Space Grows Tuesday, November 27, 12
  • 78. Compute Effort as Observation Space Grows 57->55 87->61 36->34 54->29 83->72 82->54 55->1 Tuesday, November 27, 12
  • 79. Compute Effort as Observation Space Grows 57->55 87->61 36->34 54->29 83->72 82->54 55->1 Tuesday, November 27, 12
  • 80. More Data, Less Compute Tuesday, November 27, 12
  • 82. Lessons Learned The last piece was almost as fast as the first. Tuesday, November 27, 12
  • 83. Lessons Learned The last piece was almost as fast as the first. Deep reflection (batch-based pattern discovery) was significantly more important than I had thought. Tuesday, November 27, 12
  • 84. Puzzling Experiment #3 “Adults” Tuesday, November 27, 12
  • 86. SIBOS Conference 2011 § 100 executives, 10 teams Tuesday, November 27, 12
  • 87. SIBOS Conference 2011 § 100 executives, 10 teams § 10 puzzles, 10 small tables Tuesday, November 27, 12
  • 88. SIBOS Conference 2011 § 100 executives, 10 teams § 10 puzzles, 10 small tables § Duplicate and missing pieces Tuesday, November 27, 12
  • 89. SIBOS Conference 2011 § 100 executives, 10 teams § 10 puzzles, 10 small tables § Duplicate and missing pieces Lessons: Tuesday, November 27, 12
  • 90. SIBOS Conference 2011 § 100 executives, 10 teams § 10 puzzles, 10 small tables § Duplicate and missing pieces Lessons: 1. They learned federated search bites. Tuesday, November 27, 12
  • 91. SIBOS Conference 2011 § 100 executives, 10 teams § 10 puzzles, 10 small tables § Duplicate and missing pieces Lessons: 1. They learned federated search bites. 2. I watched as an early bias misdirected their attention … but then over time new observations corrected this bias. Tuesday, November 27, 12
  • 92. Puzzling Experiment #4 “Not an Experiment” Tuesday, November 27, 12
  • 103. Puzzling Project #4: Commentary Tuesday, November 27, 12
  • 104. Puzzling Project #4: Commentary § Despite having only 100 pieces and eight collaborating eyeballs: – We began to suspect there were missing and random pieces – We had an alarming number of false positives1 – It took significantly more effort/time than expected 1 The primary source being one overly intoxicated pipeline. Tuesday, November 27, 12
  • 105. Puzzling Project #4: Commentary § Despite having only 100 pieces and eight collaborating eyeballs: – We began to suspect there were missing and random pieces – We had an alarming number of false positives1 – It took significantly more effort/time than expected § Why? Common shapes and that vast purple haze (lots of ambiguity). 1 The primary source being one overly intoxicated pipeline. Tuesday, November 27, 12
  • 106. Experiment #4: Notes to Self Tuesday, November 27, 12
  • 107. Experiment #4: Notes to Self § Excessive ambiguity drives computational cost way up Tuesday, November 27, 12
  • 108. Experiment #4: Notes to Self § Excessive ambiguity drives computational cost way up Tuesday, November 27, 12
  • 109. Experiment #4: Notes to Self § Excessive ambiguity drives computational cost way up § Some drunk people get unreasonably optimistic Tuesday, November 27, 12
  • 111. My Recommendations § Context Accumulation – Investments in general purpose information fusion will often yield greater value than investments in specialized, single-sensor, algorithms (pixel analytics). Tuesday, November 27, 12
  • 112. My Recommendations § Context Accumulation – Investments in general purpose information fusion will often yield greater value than investments in specialized, single-sensor, algorithms (pixel analytics). § Real-time Analytics Over Big Data – If something can be engineered for real-time, do that. There is a competitive advantage when one can respond intelligently while a transaction is still happening. Tuesday, November 27, 12
  • 113. My Recommendations § Context Accumulation – Investments in general purpose information fusion will often yield greater value than investments in specialized, single-sensor, algorithms (pixel analytics). § Real-time Analytics Over Big Data – If something can be engineered for real-time, do that. There is a competitive advantage when one can respond intelligently while a transaction is still happening. § Deep Reflection – Do not underestimate the value of periodic deep reflection (pattern discovery). Do this more often. And put this emerging insight to immediate use via feedback loops. Tuesday, November 27, 12
  • 114. Related Blog Posts Puzzling: How Observations Are Accumulated Into Context Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of A Pixel Data Finds Data Big Data. New Physics. General Purpose Sensemaking Systems and Information Colocation Data Beats Math And Easy to Reach Email: jeffjonas@us.ibm.com Twitter: http://www.twitter.com/jeffjonas Tuesday, November 27, 12
  • 115. Puzzling How Context Accumulates Jeff Jonas, IBM Distinguished Engineer Chief Scientist, IBM Entity Analytics Email: jeffjonas@us.ibm.com Blog: www.jeffjonas.typepad.com Twitter: http://www.twitter.com/jeffjonas Tuesday, November 27, 12
  • 116. Puzzling Vegas Neuschwanstein Beauty Down Home Music Cottage Garden Artwork provided by Hadley © 2009 Photo Copyright Robert © Kay Lamb Shannon, Artist © 2010 Royce B. McClure, Artist House Licensing, Minneapolis Cushman Hayes Licensed by Cypress Fine Art All Rights Reserved © 2011 Giesla Hoelscher © 2009 Ravensburger USA, Inc. Licensing © 2010 Ravensburger USA, Inc. All Rights Reserved © 2011 Ravensburger USA Inc. © 2011 Ravensburger USA, Inc. Tuesday, November 27, 12