Presentation a CCRMA, Stanford University https://ccrma.stanford.edu/events/alvaro-barbosa-ccrma-colloquium-public-sound-objects-web-real-world

1. Álvaro Barbosa
Research Center for Science
and Technology of the Arts
CITAR (UCP–Porto, Portugal)

2. Álvaro Barbosa http://www.abarbosa.org/
2000/2006 – MTG, UPF-Barcelona, Spain
http://mtg.upf.edu/
Since 2006 – CITAR, UCP-Porto, Portugal
http://artes.ucp.pt/citar/
Since June 2010 – CCRMA, Stanford-CA, USA
http://ccrma.stanford.edu/

3. Networked Music
Networked Music
(…) All Music Is networked. You can think about an Orchestra as client-
server network, where a conductor is serving visual information to the
client musicians, or a peer-to-peer networking model in an improvising
Jazz Combo, where there is no one directing, and the musicians are all
interacting, so, any performance context we can think of in some way
there is a network connecting the performers (…).
Networked Music with capital N and capital M (the kind we are talking
about) is about performance situations where traditional acoustic and
visual connections between participants are augmented, mediated or
replaced by electronically-controlled connections.
(From Jason Freeman s lecture opening at the 1st Networked Music Workshop during ICMC 2005)

4. Displaced Soundscapes
Displaced Soundscapes is a metaphorical description of the way lively
generated sounds can be perceived over the Internet.!
!
(Book) Barbosa, A. 2008. Displaced Soundscapes !
VDM Verlag (ISBN: 978-3-8364-7154-1)!
!
- Extensive Survey of NM Applications !
- Research on Latency and Internet Acoustics!
- Models for Networked Music Systems!
- Implementation Public Sound Objects !

5. Background on Network Collaborative Practices
Latency and Networked Music
In a vocal conversation it is possible to maintain it even with one-way
delays of up to 500 ms.
Holub, J., Kastner, M., Tomiska, O. 2007
In order to maintain a synchronized and smooth musical interaction
reduces drastically to the order of tens of milliseconds.
Schuett, N. 2002
Chafe C., Gurevich M, 2004
Lago, N and Kon, F. 2004
Barbosa, A., Carôt, A. 2005
Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005
Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006
Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009
Chafe, C., Cáceres, J-P., Gurevich, M., 2010

6. Background on Network Collaborative Practices
Latency and Networked Music
For the Human ear to perceive the order of two simultaneous sounds,
they should not be displaced in time over 20ms (Hirsh, 1959)
A difficulty in discerning the order of sounds events
ê
hard to maintain a synchronous musical interaction.
The ability to perform music synchronously is strongly dependent on:
• music style (rhythmic, melodic,…)
• musicians background and experience
• visual or physical feedback
• Sound pitch, loudness and particularly timbre (Bregman 1990)

7. Background on Network Collaborative Practices
Latency and Networked Music
What Happens?

8. Background on Network Collaborative Practices
Latency and Networked Music
Basic Principles
(1) Ensemble Performance Threshold (EPT)
(2) Echo Feed-Back (Self Delay)
(3) Inverse Proportion to Tempo
(4) Slow Attack Times
(5) Reverb and Complementary Modalities

9. Background on Network Collaborative Practices
(1) Ensemble Performance Threshold (EPT)
(CCRMA 2004, USC 2005)
Comfort Zone for Expert Musicians EPT Interval for Expert Musicians
Comfort Zone for EPT Interval for
Untrained Users Untrained Users
Almost
Extremely Impossible to
Difficult to Compensate
Unnaturally Difficult to Compensate
Low Latency Compensate
DELAY (ms) 0 5 10 15 20 25 30 35 40 45 50 75 100 150

10. Background on Network Collaborative Practices
(2) Echo Feed-Back (USC 2005)

11. Background on Network Collaborative Practices
(2) Echo Feed-Back (UCP-Porto 2005)

12. Background on Network Collaborative Practices
(3) Inverse Proportion to Tempo (UCP-Porto 2005)

13. Background on Network Collaborative Practices
(3) Inverse Proportion to Tempo (CCRMA/Banff 2006)
St. Lawrence
String Quartet
(over 50ms Delay)

14. Background on Network Collaborative Practices
(4) Slow Attack Times
Bregman (1990)
Auditory Scene Analysis

15. Background on Network Collaborative Practices
(5) Reverb and Complementary Modalities
Studies on the effect of Reverberation
(Farner 2009)
The Influence of Delay and Various Acoustic Environments
Studies on the effect of Visual Feed-Back
Pilot experiments at CCRMA since 2006
Studies on the effect of Haptic Feedback ???

16. Latency at the global level
Jitter and delay asymmetry introduce further disruption in Network
Communication.
Furthermore, Latency is not a technological condition that can be
overcome in the near future. Consider mobile or satellite technology or a
communication setup that spans worldwide.
Distance = 14.141 Km
Data Transfer = Light Speed
Latency= 94,3 ms

17. Applications for Collaborative Music
Based on Tom Rodden s CSCW Classification Space (1992)
Computer-Supported Cooperative Work for music Applications | Álvaro Barbosa, 2006

18. Shared Sonic Environments
The Shared Nature of the Internet
• Going beyond the paradigm in which we connect two
or more remote spaces for a performative session by
creating a shared space in a computer network,
inside which users can achieve a certain degree of
immersion and flexibility in their behaviour
• Users can freely join or leave this space, choosing to
participate or just to listen to an on-going acoustic
piece
• It is not oriented towards a time limited event scenario

19. Shared Sonic Environments
Shared Sonic Interface on the Web
A musical instrument that can be played using only a web browser,
enabling multiple performers to play the instrument in a collaborative way.
Jorge Herrera
CCRMA, 2009
The HORGIE:
Collaborative Online Synthesizer

20. Public Sound Objects
From the Web to the Real World

21. Public Sound Objects: 2004
Original PSOs – 2004/2006, MTG (BCN)

22. Public Sound Objects: 2004
Original PSOs – 2004/2006, MTG (BCN)

23. Public Sound Objects
PSOs CLIENTS
WEB BROWSER
Streaming Audio Client
INTERNET
PSOs SOUND
Controller Interface SERVER OBJECTS
DATA-BASE
ICECAST Streaming Server
STREAMING
AUDIO SERVER
WEB BROWSER
Apache + Custom Developed Servlet
Streaming Audio Client
Controller Interface
HTTP-SERVER
Pure-Data
INTERACTION
SERVER
(...) Pure-Data + GEM
LOCAL VISUAL Pure-Data
WEB BROWSER REPRESENTATION
Streaming Audio Client ENGINE
Controller Interface
Performance Commands
(Discrete Connection triggered by client events)
Global Audio Performance Public Installation Site
(Continuous Streaming Connection)

24. Public Sound Objects: 2000
Small Fish (Fujihata e Furukawa 1999)
Computer-Supported Cooperative Work for music Applications | Álvaro Barbosa, 2006

25. Public Sound Objects: 2005
Computer-Supported Cooperative Work for music Applications | Álvaro Barbosa, 2006

26. Public Sound Objects: 2005
Computer-Supported Cooperative Work for music Applications | Álvaro Barbosa, 2006

27. Public Sound Objects

28. Public Sound Objects: 2007
PSOs Interface
It is based on Pitch, Dynamics, Tempo and Timbre.
It tolerates Latency because:
• The resulting soundscape is musical. Yet, it does not require a
strict synchronization between sound events
• Behavior driven interactive Interface (loose coupling interaction)
• Adaptive speed and dynamics (to network conditions)
• Individual Delayed Feed-Back (same piece to everyone)
• Coherent with visual cues (spatialization of sound helps)

29. Public Sound Objects: Panorama Cues
t1 t2 t3
R
L
t1 t2 t3 Time
t1 t2 t3
t1 t2 t3
R
R
L L
t2 t3 t2 t3
t1 Time t1 Time

30. Public Sound Objects: 2007
Local Network of PSOs – CITAR (Porto)
commissioned by Casa da Musica
http://en.wikipedia.org/wiki/Casa_da_musica

31. Public Sound Objects: 2007

32. Public Sound Objects: 2007

33. Public Sound Objects: 2007

34. Public Sound Objects: 2007

35. Public Sound Objects: 2007
Client Server

36. Public Sound Objects: 2007

37. Public Sound Objects: 2007

38. Public Sound Objects: 2007

39. Public Sound Objects: 2007

40. Public Sound Objects: 2007

41. Public Sound Objects: 2007

42. Public Sound Objects: 2008

43. Public Sound Objects: 2009
Permanent Installation
Casa da Música (Renaissance Hall)

44. Thank You!!!!
abarbosa@ccrma.stanford.edu
or
abarbosa@porto.ucp.pt
http://www.abarbosa.org/pso/