Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Próximos SlideShares
Carregando em…5
×

# Content without context is meaningless

6.171 visualizações

• Full Name
Comment goes here.

Are you sure you want to Yes No
• good

Tem certeza que deseja  Sim  Não
Insira sua mensagem aqui

### Content without context is meaningless

1. 1. © Ramesh Jain Ramesh Jain and Pinaki Sinha Department of Computer Science University of California, Irvine jain@ics.uci.edu Content Without Context is Meaningless
2. 2. © Ramesh Jain Brave New Ideas  On a Nagging Old Problem – that won’t go away.  About the dead elephant in the room.  Or about the Emperor’s new clothes. “If you keep doing what you’ve always done, you’ll keep getting what you’ve always got.” This Presentation
3. 3. © Ramesh Jain Semantic Gap Discussions Semantic Gap remains strong. Lets stop ignoring the Elephant in the room. Our approaches have completely failed for the last two decades.
4. 4. © Ramesh Jain What is our Goal? 1. Using Intensity values to bridge the Semantic Gap. Or 2. Bridging the Semantic Gap in (multimedia) data. Or 3. Publishing another paper in ACM Multimedia (or CVPR).
5. 5. © Ramesh Jain You will be disappointed in this presentation if you are interested in 1. Using Intensity values to bridge the Semantic Gap. Or 3. Publishing another paper in ACM Multimedia (or CVPR).
6. 6. © Ramesh Jain Reduction: Popular Research Approach 1  X is an important problem, let me work on it.  X is too complex. Simplify it to X’.  X’ is too complex. Simplify it to X2 ’.  …  X99 ’ can be formally defined, rigorously explored, and clearly solved. Solve it.  Publish it in a conference with less than 16% acceptance rate.  Announce that X is a solved problem.
7. 7. © Ramesh Jain Data Discovery: Popular Research Approach 2  Your algorithm fails on a real data set D.  Select a more relevant data set D1. Fails again.  Carefully select a subset of Flickr data. Still Fails.  …  Select a better dataset - Coral Photos, or create a set and somehow make it Work.  Run several experiments – have several graphs so reviewers don’t complain.  Publish it in a conf. with < 16% acceptance rate.
8. 8. © Ramesh Jain Back to Content and Context Content Meaning or message: The meaning or message contained in data, as distinct from its appearance, form, or style. Context Surrounding conditions: The circumstances or events that form the environment within which something exists or takes place.
9. 9. © Ramesh Jain Falling Tree and George Berkeley  "If a tree falls in a forest and no one is around to hear it, does it make a sound”  "No. Sound is the sensation excited in the ear when the air or other medium is set in motion.“  Observation, Reality, and Perception.
10. 10. © Ramesh Jain Data Streams are all around us.
11. 11. © Ramesh Jain Data Streams are Omnipresent Visual DATA Audio DATA Text DATA Location DATA EXIF DATA Time Line DataType
14. 14. © Ramesh Jain The Challenge Connecting
15. 15. © Ramesh Jain Bits and Bytes Alphanumeric Characters Lists, Arrays, Documents, Images … Transformations
16. 16. © Ramesh Jain Semantic Gap The semantic gap is the lack of coincidence between the information that one can extract from the (visual) data and the interpretation that the same data have for a user in a given situation. A linguistic description is almost always contextual, whereas an (image) may live by itself. Content-Based Image Retrieval at the End of the Early Years Found in: IEEE Transactions on Pattern Analysis and Machine Intelligence Arnold Smeulders , et. al., December 2000
17. 17. © Ramesh Jain M ultim edia R esearcher Hammer  "To a man with a hammer, everything looks like a nail." — Mark Twain Where is the Semantic Gap? I got my Machine Learning Hammer.
18. 18. © Ramesh Jain Learning from Birds  The birds flying at the tips and at the front are rotated in a timely cyclical fashion to spread flight fatigue equally among the flock members.  The formation makes communication easier and allows the birds to maintain visual contact with each other.
19. 19. © Ramesh Jain Data is just that --- DATA, but Some of us have introduced a caste system in multimedia!!! Video Meta Data Audio Text Multimedia Caste SystemTraditional Caste System Priest Warrior Traders Who Cares
20. 20. © Ramesh Jain Modeling the World  Data  Objects  Events
21. 21. © Ramesh Jain Events and Objects  Exist in the real world.  Captured using different sensory mechanism.  Each sensor captures only a limited aspect.  Are used to understand a Situation.
22. 22. © Ramesh Jain Events take place in the real world. Events result in Data and Documents
23. 23. © Ramesh Jain Events are ‘Connectors’ Events create ‘Context’ SpatialCausal Experiential Informational TemporalStructural People Things Places Time Experiences Events Events Connect: Events are represented using 6 facets.
24. 24. © Ramesh Jain We are in the midst of the greatest Media Explosion.
25. 25. © Ramesh Jain Dealing with Photo Explosion  3 Billion Photos uploaded on Facebook – every month.  We know that most people in the world are not yet addicted to the Web, but do take photos.
26. 26. © Ramesh Jain Context Starts Before the Photo is Taken  Where  When  Why  Who (Photographer)  Which device  Parameters of the device
27. 27. © Ramesh Jain Types of Context  Context in Content  Device Parameters  Data Acquisition Context  Perceiver  Interpretation Context
28. 28. © Ramesh Jain Context in Content  Relationship among different objects and even in their subparts in real world can be utilized in analysis of data.  This has been studied since early days in computer vision and is being rediscovered once again.
29. 29. © Ramesh Jain Device Parameters  Environmental parameters of the digital devices at the time of photo taking may play key role.  What is a camera?
30. 30. © Ramesh Jain Data Acquisition Context  Knowledge about the person taking photos, location, and environmental conditions at the time of photo acquisition (e.g., sun angle, cloudy, rainy, night, indoor, etc.) affect the content of the image.
31. 31. © Ramesh Jain Where am I?
32. 32. © Ramesh Jain Perceiver  The knowledge and personality of the perceiver play a key role in interpretation of data. Knowledge Personality: Rohrsach Tests
33. 33. © Ramesh Jain Interpretation Context  Real world situation in which the data is interpreted results in focus on different aspects of the data.
34. 34. © Ramesh Jain What is a Camera?  Kodak ‘Moment’.  EXIF data is all metadata related to the Event. Exposure Time Aperture Diameter Flash Metering Mode ISO Ratings Focal Length Time Location Face
35. 35. © Ramesh Jain Sony CyberShot DSC-T2 Touchscreen 8MP Digital Camera with Smile Detection Camera = Event Capture Device
36. 36. © Ramesh Jain Meta Data for photos = Event Data
37. 37. © Ramesh Jain A new problem statement for media analysis  Given apriori knowledge about the world (concepts and relations between concepts)  Given a priori knowledge about events (lattice of events)  Given as contextual information  A set of data/ media  A possibly empty sets of tags (natural language descriptions)  The current event type to be instantiated (possibly to be first discovered)  Identify the entities occurring in the event, as described by media, data and text  with >98% precision and recall  with minimal/ acceptable user involvement Fausto Giunchiglia, DISI, University of Trento
38. 38. © Ramesh Jain Annotation Sources Calendars Annotations Environment Conditions Application Knowledge Geographic Landmarks Camera Parameters
39. 39. © Ramesh Jain Annotation Process Calendars Annotations Environment Conditions Application Knowledge Geographic Landmarks Camera Parameters Keynote Talk at ACM MM 2010, Speaker: Duncan WattsLocation: Palazzo dei Congressi EXIF: Indoor Scene; Face Keynote Talk at ACMMM10 by Duncan Watts on Using the Web to do Social Science
40. 40. © Ramesh Jain Semantic Gap is a Tough Problem – Lets Fight with it using all tools. Fighting with a Monster.