Lightning Talk at Sun's Mobile and Embedded Developer Days, January 2009

1. Just Say “Britney Spears”:
Multi-Modal Search in On Device Portals
Gail Rahn Frederick
Medio Systems, Inc.
January 2009
M3DD 2009

2. Agenda
• Introduction
• On Device Portals (ODPs)
• Multimodal ODP
• Medio Universal Search ODP
• Solving Development Challenges
• Live Demonstration
– Medio Universal Search ODP on J2ME Featurephone
Copyright 2009 Medio Systems, Inc. 2

3. Introduction
Gail Rahn Frederick
gail@medio.com or 206-262-3719
Mobile Software Architect at Medio Systems, Inc.
– Mobile Search, Advertising and Analytics
• Universal Handset Support
– J2ME, BREW, Windows Mobile, Android, iPhone, RIM, etc…
– WAP, Mobile AJAX, HTML4 on Mobile Devices
– 1600+ handset models
– Live on 9 mobile operators in 6 countries
• t-zones J2ME ODP for T-Mobile USA
– 22% Market Share in Mobile Search
• Microsoft has 6% share
– Site Search Provider for Leading Mobile Publishers
Copyright 2009 Medio Systems, Inc.

4. On Device Portals
• Handset-resident mobile application
– Preloaded or free download
• Enhances discovery and consumption
of endorsed mobile content and services
• Search, browse and purchase
• Provides a compelling user experience
– Overcomes / circumvents browser flaws
– Targets mass-market featurephones
• Personalized experience
• Sometimes monetized with advertising
Copyright 2009 Medio Systems, Inc.

5. Multimodal ODP
• Many Ways to Use an ODP
– Search, Browse, Discover, Purchase
• Many Search Modes in ODP
– Text, Voice, Bar Code, Image
• Navigate an ODP using
familiar actions on a mobile device
– saying a phrase
– entering text
– snapping a photo
Copyright 2009 Medio Systems, Inc.

6. Medio Universal Search ODP
• Consistent UI/UX across All Mobile
Platforms
– Native App, WAP and Mobile Web
• Multi-modal, search-driven J2ME ODP
– Drives repeat customer usage
– Increases usage of browsable and consumable
mobile content
– Consistent UI/UX despite J2ME fragmentation
– Search by text, voice, image and barcode
– Also available on other mobile platforms and mobile
web
• Integrated Advertising
– Leaderboards
– Interstitials
– Sponsored Links
Copyright 2009 Medio Systems, Inc.

7. Solving Challenges with Voice Search
• Sound Capture
J2ME Platforms require JSR-135 with sound record.
Requires AMR or other low-bitrate codec.
• Environment
Mobile environments are typically noisy: utterances come from cars,
bars, airports... rarely from quiet home or office environments.
• Complex Grammars
Must be efficient at handling unstructured/natural utterances
– Past: Walled-Garden ODP grammars includes song artists and titles, wallpaper titles,
game titles and studios, etc.
– Present and Future: Web-Enabled ODPs support millions of music tracks, mobile
site names, URLs, free-text queries – requires flexible voice inputs
– Consider “grammar-less” voice to text translation using phonemes.
Copyright 2009 Medio Systems, Inc.

8. Solving ODP Development Challenges
• Operator Integration
– Handoffs to operator services to preview and purchase content
• HTTP 302 redirects, browser launches, J2ME app launches
• Network Efficiency
– Compact binary protocol (See M3DD 2008 Poster by Medio)
– Piggyback status updates with search requests
– Asynchronous network communication
– Dynamic UI updates to refresh links, pages, chrome and
advertisements
• J2ME Implementation Bugs
– Share MIDlet with failed test case with operator and OEM
• Prove KVM bug with simple, commented code
• Three-way communication (ODP vendor, operator, OEM)
Copyright 2009 Medio Systems, Inc.

9. Live Demonstration
Demo Devices Sample queries
• Nokia 6263 • Britney Spears Ringtones
– T-Mobile USA 3G • Barack Obama Inauguration
– Series 40 5th Edition • Nordstrom in Santa Clara,
– MIDP 2.1, CLDC 1.1 California
– JSR 135 for Voice Recording • Facebook
– Released in Dec 2007 • Kate Winslet and Leonardo
– Price: FREE DiCaprio
• Santa Clara Weather
• Blackberry 8120 (Pearl)
– See me for demo
Copyright 2009 Medio Systems, Inc.

10. Just Say “Britney Spears”:
Multi-Modal Search in On Device Portals
Gail Rahn Frederick
Medio Systems, Inc.
January 2009
M3DD 2009