O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Video Game Music Overview

5.450 visualizações

Publicada em

Video Game Music overview for Universite de Montreal class on music and the moving image.

Publicada em: Diversão e humor, Esportes
  • Seja o primeiro a comentar

Video Game Music Overview

  1. 1. Game audio Karen Collins
  2. 2. OUTLINE <ul><ul><li>Defining Games Audio </li></ul></ul><ul><ul><li>History </li></ul></ul><ul><ul><li>Functions of Games Audio </li></ul></ul><ul><ul><li>Approaches to composition </li></ul></ul><ul><ul><li>diegesis and dynamic audio </li></ul></ul>
  3. 3. Who cares about games? <ul><li>Majority of people in western world play games (75% of heads of households (in the United States) play computer or video games)‏ </li></ul><ul><li>average age for a game player is thirty </li></ul><ul><li>nearly half of game players are female (55% male, 43% female). </li></ul><ul><li>“ Women over the age of 18 represent a greater portion of the gameplaying population (28%) than boys from ages 6 to 17 (21%). </li></ul>
  4. 4. Who cares about game audio? <ul><li>83% of Adult Gamers listed sound as one of the most important video game console elements </li></ul><ul><li>47% of Game Console owners (18-25 yr) hook up their game console to a home theatre system </li></ul><ul><li>48% of Hardcore Gamers said surround sound is a purchase driver for next generation consoles </li></ul>
  5. 5. What is game audio? <ul><li>Dialogue/speech sounds </li></ul><ul><li>Sound effects </li></ul><ul><li>Music/score </li></ul><ul><li>Ambience </li></ul><ul><li>Interface sounds (menu, etc.)‏ </li></ul>
  6. 6. What’s wrong?
  7. 7. How games are not movies: The problems of games audio <ul><li>Linearity Vs. Non-Linearity/ unpredictability, mixing </li></ul><ul><li>Interactivity with player </li></ul><ul><li>Temporality (length)‏ </li></ul><ul><li>Budgets </li></ul><ul><li>Delivery methods/technology </li></ul>
  8. 8. What is Interactive Audio? <ul><li>sound events occur in direct reaction to a player’s movements. The player triggers the cue , and can repeatedly activate it, such as by making a character jump up and down. </li></ul>
  9. 9. Interactive Audio: Footsteps Enemy music cue
  10. 10. What is adaptive audio? <ul><li>“ adaptive” audio is generally referred to as sound that reacts to transformations in the gameplay environment—such as a change from day to night set by a game’s timer mechanism . Adaptive audio is not directly triggered by a player </li></ul>
  11. 11. Adaptive Audio Change in music day to night Wolf, crow
  12. 12. What is Dynamic Audio <ul><li>Dynamic audio is a broad term to refer to both interactive and adaptive audio </li></ul>
  13. 13. <ul><li>A brief history of game audio </li></ul><ul><ul><li>(in 4 minutes!)‏ </li></ul></ul>
  14. 14. History
  15. 15. Functions of Games Sound <ul><li>Commercial functions </li></ul><ul><li>Structural reinforcement </li></ul><ul><li>Immersion/suspension of disbelief </li></ul><ul><li>Semiotics functions </li></ul><ul><li>Narrative functions </li></ul><ul><li>Emotional functions </li></ul><ul><li>Aesthetic functions </li></ul><ul><li>Kinetic functions </li></ul>
  16. 16. 1. Commercial functions <ul><ul><li>Sell games </li></ul></ul><ul><ul><li>Sell artists </li></ul></ul><ul><ul><li>Sell soundtracks </li></ul></ul><ul><ul><li>Sell movies </li></ul></ul><ul><ul><li>Sell other games </li></ul></ul><ul><ul><li>Branding: familiarization </li></ul></ul>
  17. 17. Sales, marketing and branding
  18. 18. 2. Structural reinforcement <ul><li>Mask structure and editing </li></ul><ul><li>Enhance/reinforce structure of game </li></ul><ul><li>Create structure </li></ul><ul><li>Situate player in game matrix </li></ul><ul><li>Indicate change in narrative </li></ul><ul><li>Enhance continuity </li></ul>
  19. 19. Structural reinforcement
  20. 20. 3. Immersion/suspension of disbelief <ul><li>Add realism: </li></ul><ul><ul><li>Emphasize real sounds </li></ul></ul><ul><ul><li>Represent location/spatialization </li></ul></ul><ul><li>Create illusion </li></ul><ul><li>Cover external noise </li></ul>
  21. 21. Immersion/suspension of disbelief
  22. 22. 4. Semiotic functions <ul><li>Sonic symbols, leitmotifs: identify objects and decrease learning curve </li></ul><ul><li>Preparatory function </li></ul><ul><li>Acousmatic sound: Focus attention, identify goals </li></ul><ul><li>Convey meaning, emotion </li></ul><ul><li>Comment </li></ul><ul><li>Add slant or bias </li></ul>
  23. 23. Semiotic Preparatory Function
  24. 24. A test of the semiotics of space/place in Zelda <ul><li>Ranch (farm)‏ </li></ul><ul><li>Windmill </li></ul><ul><li>Fire temple </li></ul><ul><li>Evil Castle underground </li></ul><ul><li>Valley of gypsy thieves </li></ul><ul><li>Flying fairy </li></ul><ul><li>Dawn </li></ul><ul><li>Temple of time </li></ul><ul><li>Ice cavern </li></ul>A B C D E F G H I
  25. 25. 5. Narrative functions <ul><li>Advancement of plots </li></ul><ul><li>Locate player in storyline </li></ul><ul><li>Anchor player in terms of game matrix/level, etc. </li></ul><ul><li>anticipate/foreshadow </li></ul><ul><li>Dialogue can be a major event in plot </li></ul><ul><li>Access character’s thoughts/reveal character details </li></ul><ul><li>Reveal goals </li></ul>
  26. 26. Narrative: correct response
  27. 27. 6. Emotional functions <ul><li>Emotional and physiological: manipulate player </li></ul><ul><li>Disturb or enhance gameplay: battle scenes </li></ul><ul><li>Mood induction </li></ul><ul><li>Create intimacy (immersive)‏ </li></ul>
  28. 28. Emotional
  29. 29. 7. Aesthetic functions <ul><li>Improve our appreciation of the game </li></ul><ul><li>Make the game better/more enjoyable </li></ul><ul><li>Make it “cool” </li></ul><ul><li>Indicate genre/style </li></ul><ul><li>Inter-textual referencing </li></ul>
  30. 30. 8. Kinetic functions <ul><li>Aerobic exercise: fitness </li></ul><ul><li>Music as motivating factor for movement </li></ul><ul><li>Edutainment for toddlers, etc. </li></ul>
  31. 31. Kinetic Functions <ul><li>Dance Dance Revolution, etc. </li></ul>
  32. 32. Problems (and some solutions) of Interactive Audio in Games
  33. 33. Mixing: Problems: -too many sounds -no “post production” mixing -unpredictable timings : results in “muddy”, clash of sounds
  34. 34. Mixing Solutions! <ul><li>Reduce amount of sounds </li></ul><ul><li>Control frequencies of sounds </li></ul><ul><li>Assign priorities in code </li></ul><ul><ul><li>E.g. dialogue = priority “1” </li></ul></ul><ul><ul><ul><li>Ambience = priority “4” </li></ul></ul></ul><ul><ul><ul><li>Gunshot = priority “2” </li></ul></ul></ul><ul><ul><ul><li>Music = priority “3” </li></ul></ul></ul><ul><ul><ul><li>Footsteps = priority “5” </li></ul></ul></ul>
  35. 35. Problem: technological constraints <ul><li>Trying to write songs in a few kB for constrained technology such as mobile phones </li></ul>
  36. 36. Solutions: technological constraints: <ul><li>Playing a single sample at different pitches to produce variation. </li></ul><ul><li>The &quot;pitch it up, play it down&quot; technique for saving space. </li></ul>
  37. 37. Sampling to save size <ul><li>This game contains only a single &quot;trumpet fall&quot; sound effect </li></ul><ul><li>modify the playback sample rate so that each time you pick up a piece of cheese, the sound is played at a different pitch </li></ul><ul><li>randomly vary the pitch during gameplay, so there's no repeating pattern to it </li></ul><ul><li>The sound tends to mask the transition from one mix to another, helping to create a more seamless audio experience. </li></ul><ul><li>same kind of pitch-shifting effect applied to car horn beeps: (fig 1)‏ </li></ul>
  38. 38. Sampling to save space <ul><li>Another way to save space is using the &quot;pitch it up, play it down&quot; technique. </li></ul><ul><li>take your original, high-resolution sound effect and transpose it up an octave, halving the length. </li></ul><ul><li>Then convert it to a low-resolution compressed format, like this: (cat growl, octave up, car start, octave up. Then in the game, play it down an octave: (octave down). </li></ul><ul><li>Although the game sound might be a little crunchy, you've just cut the size of your file in half without losing too much audio fidelity. Obviously, the higher you pitch the sound, the &quot;crunchier&quot; the playback will become, but the technique can be used for custom instruments as well as sound effects, and is particularly effective on the tiny speakers in mobile devices. </li></ul>
  39. 39. <ul><li>Problem: transitions from cue to cue </li></ul><ul><ul><li>No “post” = unpredictable timings </li></ul></ul><ul><ul><li>Abrupt jumps are jarring </li></ul></ul>
  40. 40. (1) Solution: transitions: Cross fade
  41. 41. <ul><li>Problem: Multi-player interactivity </li></ul><ul><li>Unscripted events! </li></ul><ul><li>parameter-based music is difficult: </li></ul><ul><ul><li>Music cannot be tied to specific events or locations </li></ul></ul><ul><ul><li>Music cannot be tied to specific parameters </li></ul></ul>
  42. 42. Solutions: multi-player interactivity <ul><li>Work with multi-players! </li></ul><ul><li>Asheron’s Call II, Lord of the Rings: players can bring instruments and jam out with each other </li></ul>
  43. 43. Multi-player music
  44. 44. Problem: listener fatigue <ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul><ul><li>Loops are boring. </li></ul>
  45. 45. Solutions: Listener fatigue <ul><li>Learn how to use silence! </li></ul><ul><li>“ boredom switch”: timer for set levels to stop </li></ul><ul><li>VARIABILITY! </li></ul>
  46. 46. Dynamic Composition: Variability <ul><li>variable tempo </li></ul><ul><li>variable pitch </li></ul><ul><li>variable volume </li></ul><ul><li>variable DSP </li></ul><ul><li>variable melodies (algorithmic generation)‏ </li></ul><ul><li>Variable harmony (chord arrangements)‏ </li></ul><ul><li>Variable harmony (keys)‏ </li></ul><ul><li>variable mixing </li></ul><ul><li>Variable sequencing </li></ul><ul><li>variable structure </li></ul>
  47. 47. Variable tempo
  48. 48. Variable tempo
  49. 49. Variable pitch
  50. 50. Variable Pitch
  51. 51. Variable volume
  52. 52. Variable DSP <ul><li>Change DSP effects to change music: </li></ul><ul><ul><li>E.g. reverb to make it softer, dreamier </li></ul></ul><ul><ul><li>Phasing to create a “dazed” effect </li></ul></ul><ul><ul><li>Overdrive to make it more aggressive, etc. </li></ul></ul>
  53. 53. Variable melodies <ul><li>Ballblazer (1984): the music plays forever, without repeating itself but without straying too far from the original theme. The bassline doesn't vary at all, only the lead line in the higher register is algorithmically varied. </li></ul>
  54. 54. Variable harmony <ul><li>Asheron’s Call 2: changes in mode indicate changes in gameplay: </li></ul><ul><ul><li>A player standing alone in an area would hear the background score in Lydian, but as creatures fill the area, the music transitioned to Phrygian. If monsters outnumber player, music transition to diminished mode and quickened drum groove </li></ul></ul>
  55. 55. Variable mixing <ul><li>Adjust mix by </li></ul><ul><ul><li>Bringing instruments up in mix that were quiet or silent before </li></ul></ul><ul><ul><li>Drop elements from mix </li></ul></ul>
  56. 56. Cheese Racer <ul><li>Muting and unmuting multiple tracks in a MIDI file to produce various mixes. </li></ul><ul><ul><li>2 drum loop samples, </li></ul></ul><ul><ul><li>three tones: bass, melody and chords </li></ul></ul><ul><li>In the code for the first level, ten different combinations of tracks: percussion + bass, percussion + bass + melody, bass + melody + chords, etc. </li></ul>
  57. 57. (cont..)‏ <ul><li>Notice how the mix changes every time the mouse gets a piece of cheese. (Fig 2 : exit) (Fig 3)‏ </li></ul><ul><li>Music loop 40 seconds. 10 different combinations: therefore, each level has more than 6 minutes of different music mixes. “Because the mix changes depending on gameplay, the music will never play exactly the same way twice, thereby increasing variation and decreasing &quot;ear fatigue.&quot;” </li></ul><ul><li>In fact, there are two more sets of tracks playing two more styles of music, using the same tempo and percussion tracks as the first level. </li></ul><ul><li>Entire game contains almost 20 minutes of various music mixes, using only 68K of compressed sample and MIDI data. </li></ul>
  58. 58. Dynamic Music- example <ul><li>Layer of instrumentation added when Mario rides Yoshi in Super Mario World. </li></ul>
  59. 59. Russian Squares <ul><li>Using layers, each cue increment represents a puzzle row being cleared by the player. </li></ul>
  60. 60. Variable sequencing <ul><li>Alter sequences of music </li></ul><ul><li>Zelda Ocarina: Hyrule field </li></ul>
  61. 61. Variable structure <ul><li>Create “transition matrix”: chart out all possible directions for each sequence, and create transitions for marker points/jumps </li></ul>
  62. 62. Transition Matrix: No One Lives Forever <ul><li>Music and Arranging by Guy Whitmore and Tobin Buttram </li></ul><ul><li>This music clip moves through all six of the music states for the Ambush theme, demonstrating transitions called from a transition matrix. Time      Music State 0:00      6 (combat 2) 1:22      transition 1:30      2 (sub-ambient) 2:14      transition 2:16      4 (suspense) 3:10      transition 3:12      3 (ambient) 4:08      transition 4:12      5 (combat 1) 5:33      transition     1 (silence)‏ </li></ul>
  63. 63. No One Lives Forever <ul><li>EarthOrbit : </li></ul><ul><li>The Ambush theme starts in music state 5 (combat 1), transitions to music state 2 (ambient), then transitions to music state 6 (combat 2) with motifs. </li></ul>
  64. 64. What else can be done? <ul><li>Dynamic music offers all kinds of interesting possibilities to have music as a driving force in a game! </li></ul>
  65. 65. Super Mario DS <ul><li>New Super Mario Bros DS </li></ul><ul><ul><li>Enemies jump and fly in time to music </li></ul></ul>
  66. 66. Electroplankton
  67. 67. Rez
  68. 68. Vib Ribbon
  69. 69. Levels of dynamic sound <ul><li>non-dynamic, non-diegetic sound </li></ul><ul><li>Adaptive non-diegetic sound </li></ul><ul><li>Interactive non-diegetic sound </li></ul><ul><li>Non-dynamic diegetic sound </li></ul><ul><li>Adaptive diegetic sound </li></ul><ul><li>Interactive diegetic sound </li></ul><ul><li>Kinetic diegetic and non-diegetic </li></ul><ul><li>(reminder: Non-Diegetic: Outside the character’s space. Diegetic: “real sounds”, inside character’s space)‏ </li></ul>
  70. 70. Non-dynamic, non-diegetic sound <ul><li>non-dynamic, linear music or sound found most frequently in the cinematic sequences (movie sequences in which the player’s functions are stopped while a short clip to advance the plot is played). In other words, the player has no control over the possibility of interrupting the music (short of resetting/turning the game off). </li></ul>
  71. 71. Non-dynamic, non-diegetic sound
  72. 72. Adaptive non-diegetic sound <ul><li>Sound events occurring in reaction to gameplay, but which are unaffected by the player’s direct movements </li></ul><ul><li>The player cannot re-trigger these events. </li></ul>
  73. 73. Adaptive non-diegetic sound
  74. 74. Interactive non-diegetic sound <ul><li>Sound events occurring in the character’s space </li></ul><ul><li>The player can re-trigger these events. </li></ul>
  75. 75. Interactive non-diegetic sound
  76. 76. Non-dynamic diegetic sound <ul><li>In non-dynamic diegetic audio , the sound event occurs in the character’s space, but the character does not directly interact with it. </li></ul>
  77. 77. Non-dynamic diegetic sound
  78. 78. Adaptive diegetic sound <ul><li>Diegetic sounds adapt to gameplay environment (e.g. night to day)‏ </li></ul>
  79. 79. Adaptive Diegetic Sounds
  80. 80. Interactive diegetic sounds <ul><li>occur in the character’s space and with which the player’s character can interact. The player instigates the audio cue, but does not necessarily affect the sound of the event once the cue is triggered. </li></ul>
  81. 81. Interactive Diegetic Sounds
  82. 82. Kinetic interactive Sound <ul><li>Player interacts with the sound by mimicking the sound/image kinetically </li></ul><ul><ul><li>Guitar Hero </li></ul></ul><ul><ul><li>Donkey Konga </li></ul></ul>