O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

'Hear' & Now: Software Integration for the Qualcomm Snapdragon Audio Feature

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
1©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
‘Hear’ & Now
Qualcomm® Snapdragon™ Audio
Ravi Satyanarayanan, Director Product Management, Qualcomm Technologies, Inc.
Ton...
3©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
‘Hear’ & Now: Snapdragon Aud...

Vídeos do YouTube não são mais aceitos pelo SlideShare

Visualizar original no YouTube

Carregando em…3
×

Confira estes a seguir

1 de 43 Anúncio

'Hear' & Now: Software Integration for the Qualcomm Snapdragon Audio Feature

Baixar para ler offline

The session combines live demos, how-to info and third-party presentations on how they’ve taken advantage of powerful audio capabilities in the Qualcomm® Snapdragon™ processor, a product of Qualcomm Technologies, Inc. Learn how to use hardware/software integration to provide pro-quality audio processing solutions customized for differentiated listening experiences.

View this presentation on YouTube: https://www.youtube.com/watch?v=bPrqdqUVyyw&list=PLxeazpXYyqtNm2EnCbfSzy7aKOkHjiaSi&index=18

The session combines live demos, how-to info and third-party presentations on how they’ve taken advantage of powerful audio capabilities in the Qualcomm® Snapdragon™ processor, a product of Qualcomm Technologies, Inc. Learn how to use hardware/software integration to provide pro-quality audio processing solutions customized for differentiated listening experiences.

View this presentation on YouTube: https://www.youtube.com/watch?v=bPrqdqUVyyw&list=PLxeazpXYyqtNm2EnCbfSzy7aKOkHjiaSi&index=18

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a 'Hear' & Now: Software Integration for the Qualcomm Snapdragon Audio Feature (20)

Anúncio

Mais de Qualcomm Developer Network (20)

Mais recentes (20)

Anúncio

'Hear' & Now: Software Integration for the Qualcomm Snapdragon Audio Feature

  1. 1. 1©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
  2. 2. ‘Hear’ & Now Qualcomm® Snapdragon™ Audio Ravi Satyanarayanan, Director Product Management, Qualcomm Technologies, Inc. Tony Rodriguez, CTO, Digimarc Richard Kjerstadius, Director, DSP Software, Opalum Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc.
  3. 3. 3©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. ‘Hear’ & Now: Snapdragon Audio User Experience 1 Ecosystem Enablement 2 Digimarc Watermark Detection 3 Opalum Speaker Protection 4
  4. 4. 4©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Audio user experience Enhancing and expanding the phone experience Camcorder Entertainment Natural user interaction Voice call Music playback
  5. 5. 5©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Voice call. Hear. And be heard. Voice enhancement technology for clearer conversations ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. • Fluence & Fluence Pro noise cancellation technology echo cancellation and noise suppression • Freedom to hold the phone how you want − Landscape or portrait speakerphone − Handset • Hear your conversation & not the background sound − Active Noise Cancellation to suppress noise around you − Automatically amplify speech relative to background sound • HD Voice in all calls • Reduce noise on the incoming calls
  6. 6. 6©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Music playback Hear music as the musicians intended 6 • Hi-Fi audio at the same quality as studio masters - High quality WCD codec - End-to-end 24bit / 192kHz • Listen all day with low power playback on the DSP • Active Noise Cancellation to hear only what you want to hear ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
  7. 7. 7©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Home theatre on-the-go Enjoy the same surround sound available in home theaters wherever you are 7 • Virtual surround sound over headphones to bring the immersive experience with you • Big booming sound - Bigger bass, clearer vocals, louder sounds • Collaborations with industry leaders: Dolby, DTS ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
  8. 8. 8©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Camcorder Capture what you want to hear • Capture what you see with Sound Focus - Suppress unwanted noise from outside of the field of view • Audio zoom when the camera zooms in • Surround Sound Record for the Full HD / 4K camcorder experience • High quality audio, with high dynamic range ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
  9. 9. 9©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Voice natural user interface • Always on, low power voice activation - No need to press a button to interact with your device - Launch applications and give commands quickly • Activate & Unlock in one step - No need to enter a PIN • Hands free operation with Fluence - Drive mode ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved.
  10. 10. 10©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. ‘Hear’ & Now: Snapdragon Audio User Experience 1 Ecosystem Enablement 2 Digimarc Watermark Detection 3 Opalum Speaker Protection 4
  11. 11. 11©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Audio ecosystem • Added on Snapdragon 810, 808 • Great for low latency, timing sensitive algorithms • Speaker Protection by Opalum Audio front-endVoice Audio 3rd-party relationships for OEM differentiation • End-to-end high-fidelity audio – Native 24bit/192kHz framework • Low power decode on Qualcomm® Hexagon™ DSP • Snapdragon Voice Activation • ISVs can enable similar solutions using the framework Always on, low power • DSP-accelerated voice codecs and ECNS for VoIP • Hexagon Access Program support to add voice codecs and processing modules Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.
  12. 12. 12©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. How others add value, and find success
  13. 13. 13©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. ‘Hear’ & Now: Snapdragon Audio User Experience 1 Ecosystem Enablement 2 Digimarc Watermark Detection 3 Opalum Speaker Protection 4
  14. 14. Qualcomm® Snapdragon SenseTM Audio + Digimarc Discover Enabling Low-Power Always-On Contextual Awareness Tony Rodriguez Qualcomm Snapdragon Sense is a product of Qualcomm Technologies, Inc.
  15. 15. — Slide 15 — ©2014 Digimarc Corporation Digimarc Vision and Mission • Enable mobile devices to see, hear, and understand by providing digital identities for all media objects (Internet of Everything) Approach • Lessons learned from the last decade of deployment (started w/DoJa & BrewTM): • ‘Seeing & Hearing’ is a function of how fast, accurate and reliable the underlying object/media recognition algorithms are. • Deterministic object recognition is significantly quicker, more accurate and reliable. • Developers are best served when fast, reliable and accurate object recognition is available as a system service that minimizes bandwidth consumption and is always-on & low-power. • 2013: Delivered Intuitive Computing Platform (ICP) to our retail, publishing and brand partners who have deployed apps to enable engagement with print, packaging, radio, etc. BREW is a product of Qualcomm Technologies, Inc.
  16. 16. — Slide 16 — ©2014 Digimarc Corporation Digimarc Discover built on ICP: Enabling Shoppers Journey • Digimarc® Discover app & SDK’s built on Intuitive Computing Platform. • Enables object and media recognition through all phases of Shopper’s Journey • Discovery & transaction via mobile devices across all touch points of the Journey • Digimarc® Barcode: Imperceptible barcodes for CPG that can double throughput at front-of-store.
  17. 17. — Slide 17 — ©2014 Digimarc Corporation Examples: Publications, TV, Retail Print to Digital Contextual Second-Screen Shopping
  18. 18. — Slide 18 — ©2014 Digimarc Corporation Deterministic Identification Powered by Digimarc ID’s We can’t see or hear Digimarc ID’s, but mobile and wearable devices can Digimarc ID Original Digital Media Digimarc ID Imperceptibly Embedded Enabled Content Print Music, Movies or Images Television and Radio
  19. 19. — Slide 19 — ©2014 Digimarc Corporation Comparison of App Experience in Noisy Environment • Digimarc Audio ID’s effective in hostile environments. • Accuracy maintained, no missed matches (deterministic) • Developers can assign multiple ID’s, creating richer, contextually aware experience.
  20. 20. — Slide 20 — ©2014 Digimarc Corporation Digimarc Audio ID: Details • Signal Construction • Weak signal in 120 Hz to 8 kHz range, communicated via host audio (channel) • Extensible & key based payload protocols • Embedding • Psychoacoustic principles and empirical analysis used to hide watermark • Ambient Detection • Spectral Processing • Payload Recovery Technique Frequency Domain WM Description Frequency Domain Spread Spectrum Characteristics Watermark signal added to audio spectrum (frequency magnitude) Payload Extensible 24 bit Audio required for detection 1-3 Seconds
  21. 21. — Slide 21 — ©2014 Digimarc Corporation Porting to the Qualcomm® HexagonTM Goal: People don’t have “ear-lids” – sound is the most pervasive and rich source of context in the human experience, our sub-conscious is continually (even when asleep) categorizing audio and applying policies on when and how to engage our consciousness. Mobile & wearable devices should do the same! • Phase 1: Re-factoring (pipeline architecture), Fixed Point, Memory Reduction • Phase 2: Hexagon Port of Detector • Phase 3: Low-Power Activator Implementation • Result: Developers able to create always-on contextually aware applications that are fast, reliable and accurate. Qualcomm Hexagon is a product of Qualcomm Technologies, Inc.
  22. 22. — Slide 22 — ©2014 Digimarc Corporation Phase 1: Re-factoring Detector (Pipeline Architecture) • Mobile implementation optimized for OS & application processor • Each block re- coded, reducing memory consumption and functional granularity to aide in porting • 8MB -> 256K Runtime FFT Intermediate Accumulation Magnitudes Spectral Filter Final Accumulation Magnitudes Normalize Magnitudes For every 512 samples @ 16 Khz For each 1.024 second CRCSNR Gating Decode Select Decode Candidates Version Detection For each 1.024 second For each 1.024 second + Scaling Audio Digimarc ID Spectral Processing Payload Recovery
  23. 23. — Slide 23 — ©2014 Digimarc Corporation Phase 2: Hexagon Port DetectorApplication Hexagon & Audio Codec Audio • Application reliant on High Level Operating System (HLOS) to retrieve audio samples and deliver to Detector • Audio sampled at 16Khz but processed only once a second. • Difficult for Application Processor cores to enter low- power state. Codec Application Processor Digimarc Qualcomm Technologies Hexagon SDK
  24. 24. — Slide 24 — ©2014 Digimarc Corporation Phase 2: Hexagon Port Detector StubApplication Application Processor Skel Fast RPC Stub Detector • Hexagon SDK to Port detector C++ source. • Significantly reduced load on Application Processor cores • Utilized library of optimized functions for Hexagon (FFT, etc.) • QAIC handled complexity of FastRPC marshaling Audio Codec Hexagon & Audio Codec Digimarc Qualcomm Technologies Hexagon SDK
  25. 25. — Slide 25 — ©2014 Digimarc Corporation Phase 3: Low-Power Activator Detector StubApplication Application Processor Audio Skel Fast RPC Stub Codec “Activator”Service&PolicyEngine • Worked with Qualcomm Technologies to build Activator and route audio from Codec • Eliminated need for HLOS to manage audio. Hexagon & Audio Codec Detector Digimarc Qualcomm Technologies Hexagon SDK
  26. 26. — Slide 26 — ©2014 Digimarc Corporation Phase 3: Low-Power Activator Detector StubApplication Application Processor Audio Skel Fast RPC Stub Detector Codec “Activator”Service&PolicyEngine • Worked with Qualcomm Technologies to build Activator and route audio from Codec • Eliminated need for HLOS to manage audio. • Policy Engine able to wake Application Processor and invoke appropriate application when Digimarc Audio ID detected • Network bandwidth consumed after object identification. Hexagon & Audio Codec Policy Engine Digimarc Qualcomm Technologies Hexagon SDK
  27. 27. — Slide 27 — ©2014 Digimarc Corporation Phase 3: Low-Power Activator Detector StubApplication Application Processor Audio Skel Fast RPC Stub Detector Codec “Activator”Service&PolicyEngine • Worked with Qualcomm Technologies to build Activator and route audio from Codec • Eliminated need for HLOS to manage audio. • Policy Engine able to wake Application Processor and invoke appropriate application when Digimarc Audio ID detected • Network bandwidth consumed after object identification. • Result: Application Processor & RF allowed to enter low-power state. • Developers able to create always-on contextually aware applications that leverage the ubiquity of audio in our everyday lives Hexagon & Audio Codec Policy Engine Digimarc Qualcomm Technologies Hexagon SDK
  28. 28. — Slide 28 — ©2014 Digimarc Corporation Conclusion : Snapdragon Sense Audio + Digimarc Audio ID’s • People don’t have “ear-lids” – sound is the most pervasive and rich source of context in the human experience, our sub-conscious is continually (even when asleep) categorizing audio and applying policies on when and how to engage our consciousness. • Mobile and wearable devices are now able to do the same, remain always aware and wake the Application Processor as needed. • Able to maintain an always-on posture with no network dependencies – ramps power and bandwidth consumption after identification has occurred (unique to Digimarc ID’s). • Enabled by Snapdragon Sense Audio & Digimarc Audio ID’s that provide accurate, reliable context. • Result: Accurate and application specific context can be incorporated in all sources of audio. • Applications enabled for developers: Retail, Second-Screen, Indoor Location, Radio, Concerts, etc.
  29. 29. ©2014 Digimarc Corporation
  30. 30. 30©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. ‘Hear’ & Now: Snapdragon Sense Audio User Experience 1 Ecosystem Enablement 2 Digimarc Watermark Detection 3 Opalum Speaker Protection 4
  31. 31. © 2014Opalum AB High-End audio from constrained devices
  32. 32. © 2014Opalum AB Agenda • Trends in MobileAudio • OpalumOverview andTech • Collaboration with QualcommTechnologies • What’s next?
  33. 33. © 2014Opalum AB Trends in Mobile Audio • Maximizing loudness and bass – Power doesn’t kill a speaker – Pushing the limits on thinner form-factors • Multi-channel audio goes mobile – Virtualization techniques for movies and games. – Parametric audio • Reducing power consumption – “Always on” features demand DSP core offloading – Increasing level of integration into main chipset – Other ways of reducing power consumption?
  34. 34. © 2014Opalum AB Overview andTech
  35. 35. © 2014Opalum AB
  36. 36. © 2014Opalum AB
  37. 37. © 2014Opalum AB
  38. 38. © 2014Opalum AB
  39. 39. © 2014Opalum AB
  40. 40. © 2014Opalum AB Collaborating with QualcommTechnologies • Seamless integration between LabSuite and Snapdragon • Audio tuning lead-time drastically reduced • Working with the Hexagon tools • Dynamic loading enables really cool android app possibilities
  41. 41. © 2014Opalum AB What’s Next?
  42. 42. © 2014Opalum AB For more information on Qualcomm, visit us at: www.qualcomm.com & www.qualcomm.com/blog ©2013-2014 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Qualcomm, Snapdragon, Hexagon, Brew and Fluence are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Uplinq and Snapdragon Sense iaretrademarks of Qualcomm Incorporated. All Qualcomm Incorporated trademarks are used with permission. HY-FI is a trademark of Qualcomm Atheros, Inc., registered in the United States and other countries, used with permission. Other products and brand names may be trademarks or registered trademarks of their respective owners.of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT. Thank you FOLLOW US ON:

×