This document discusses sound design for mobile apps and games. It introduces Pure Data (Pd), an open-source visual programming environment for procedural audio generation. It also introduces libpd, which allows embedding a Pd instance within iOS and Android apps. Two example apps are provided: a simple keyboard app and a space shooter game app enhanced with dynamic sound effects created using Pd patches. The document provides an overview of sound design principles and techniques for mobile as well as tutorials on basic Pd functionality and libpd integration.
2. 360 iDev tackmobile.com
About Me
• @thillerson, +thillerson (if you’re into that)
• Developer at Tack Mobile (tackmobile.com)
• iOS, Android, and Mobile Web
• Rails, Node, maybe Elixir (one day)
6. 360 iDev tackmobile.com
Sound Design Work: Adrift
• Puzzle Game - “Adrift by Tack” in App Store
• Background Music
• Sound Effects
7. 360 iDev tackmobile.com
Programming Sound
• In Beta Soon*
(*this week)
• http://pragprog.com/
• What we’re talking
about today...
• ... and much more
8. 360 iDev tackmobile.com
What We’re Going To Do Today
• Answer: What Is Sound Design?
• Discuss UX Principles of Sound Design
• Introduce Pure Data
• Introduce libpd
• Tour Two iOS Apps Embedding Pd
9. 360 iDev tackmobile.com
When We Are Done, You Will
• Understand Sound Design
• Understand Procedural Audio
• Know what Pd is, and how libpd makes it
awesome
• Be excited about Dynamic Audio in your apps
and games
12. 360 iDev tackmobile.com
A Sound Designer...
• Conceptualizes
• Captures or Designs
• Records
• Engineers
• Produces
• ... the sound for
an experience
13. 360 iDev tackmobile.com
Sound Designer Skills
• Understand Sound
• Intuitive vs. Theoretical
• Music vs. Sound
• Audio Engineering
14. 360 iDev tackmobile.com
Sound Design in Mobile Apps
• Developers/Designers wear lots of hats
• Control
• Efficiency
• Asset size
17. 360 iDev tackmobile.com
Sound Design and Good UX
• Your app doesn’t need sound
• Expect to be turned off
• Don’t make me listen to you
• Listen on lots of devices/environments
• Engineer for small speakers
• Make your sound mean something!
18. 360 iDev tackmobile.com
Always
• Test with phone calls
• Test while playing music
• Have fallbacks - e.g. No music, just effects
• Provide control to the user
19. 360 iDev tackmobile.com
A Good Example: Clear
• Realmac Software
• Josh Mobley
• Meaningful Sound
• demo
26. 360 iDev tackmobile.com
Properties of Waves
• A - Wavelength (Distance), Period (Time)
• B - Amplitude
• C - Frequency - 1/Period (measured in Hertz)
A
B
B
C 1 2 3
27. 360 iDev tackmobile.com
Fourier Series
• Based on his work ------->
• Periodic signals composed
of simple oscillating
functions
• Sine/Cosine waves are
fundamental
32. 360 iDev tackmobile.com
Procedural Audio
• Functions
• Objects
• Controls and Components
• Produce a Signal
• “Building Sound from First Principles”
33. 360 iDev tackmobile.com
Pure Data
• Procedural Audio
• Visual Programming Environment
• Free and Open Source
• pure-data.info
35. 360 iDev tackmobile.com
Signals and Values
• Signals are streams of numbers, usually audio
data
• Values are numbers or symbols
36. 360 iDev tackmobile.com
• Pd’s class
• or, kind of like a command line tool
Objects
osc~ 440
signal inlet inlet
signal outlet
class name arguments
37. 360 iDev tackmobile.com
Hot vs Cold Inlets
• Hot inlets make objects immediately act
• Cold inlets usually store a value
• Execution is right to left
38. 360 iDev tackmobile.com
• Signal connections are thick
• Others are thin
Connections
osc~ 440
*~
dac~
1
40. 360 iDev tackmobile.com
libpd
• http://libpd.cc
• Peter Brinkmann - Google Engineer
• Enables you to embed an instance of Pd in your
app
• iOS and Android
42. 360 iDev tackmobile.com
Source Code
• github.com/thillerson/dynamic-sound-ios
• Examples of capabilities, not necessarily the
best architecture
• Goal: understand capabilities, have a place to
start