This document summarizes key points from Bill Moggridge's book about designing multisensory and multimedia interactions. It discusses the work of several interaction designers including Hiroshi Ishii who developed tangible user interfaces that blend the physical and digital worlds. Durrell Bishop emphasized making digital objects self-evident like physical objects. Joy Mountford helped develop QuickTime and add video and sound to computers. Bill Gaver studied sound perception and developed auditory icons. The document also describes several prototypes like Ishii's Music Bottles, Bishop's Marble Answering Machine, and Gaver's History Tablecloth that sensor weight over time.
2. Content: 5 senses: Vision, hearing, touch, smell & taste Connection with computers “sensory deprived and physically limited” explores the opportunities for interaction design to become multisensory and to take advantage of multimedia. Interviews: Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT) Durrell Bishop (Teacher of Product Design @ Royal College of Art in London) Joy Mountford (Member of Human Interface Group @ Apple) Bill Gaver (Professor of Interaction Research @ Royal College of Art in London)
3. Vision « Visions of Japan » by Arata Isozaki Realms of Cliché(presented various entertainments(such arts as the tea ceremony, flower arrangement and popular and classical performing arts) Kitsch(examined the nature of competitiveness in games and sports) Simulation(demonstrated a dramatic expansion of the potential for visual display ) The images produced by these systems are displayed on screensbut they are completely separated from the real things themselves—processed, edited, and otherwise qualitatively changed, often intosomething completely different. “blurring the line between real and simulated”
4. Hiroshi Ishii Physical & Digital world Interaction requires 2 key components: Controls ( manipulate access to digital informations and computations) Representation(people can perceive the results of computations)
5. Hiroshi Ishii Physical & Digital world Interaction requires 2 key components: Controls ( manipulate access to digital informations and computations) Representation(people can perceive the results of computations) Intangible pixels- windows,menus,folders To control representations- remote controls,mouse Tangible representation (control mechanism) allow user to directly grap and manipulate. Dynamic output or feedback(video, sound)
6. Tangible User Interfaces Ping Pong Plus (http://www.youtube.com/watch?v=AZO8sfmpKIQ&playnext=1&list=PL38930FA142C5C68F) a ping pong table that could sense ball hits, which it used to control various visualizations projected on the table. Music Bottles http://www.youtube.com/watch?v=U4IYyNL4ld8&feature=related bottles as containers and controls for digital information bottles enable each one to be wirelessly identified simple affordance of opening and closing
7. Durrell Bishop Things should be themselves. (self-evident - to describe whatthey are actually doing) Monopoly versus Video Tape Recorder Monopoly makes the functions of the game self-evident. There is a physical representation of the different elements of the software of the game. For VTR, you might want to know whether it is recording or not, how much space there is left, or where things are.
8. Physical object represent meaning.. We remember the things, experience them repeatedly Unusual or surprising to our memory Example: coin (has value – has country, ownership) «...make virtual items, both space andobjects, seem just as real as physical items by designing themtosay more about themselves.» physical object pointer to a window or folder, thephysical environment is just as real as the screen environment
9.
10. Joy Mountford Quick Time Mountford & Mike Mills developed designs to include images and videos in graphical user interfaces, integrating QuickTime and moving the personal computer towards multimedia. Personal computers started to grow. Bringing video to personal computer turn the PC into multimedia device. Paper animation techniques was replaced by Quick Time.
11. People started to be familiar with film and video editing tools and techniques. ( edit clips of their children) Hand controller Navigable movies (became QuickTime VR ) – Golden Gate Bridge had a video camera pointing at an object, seemingly doing thesame thing day after day. You could move around the camera’s viewof the room just like you could manipulate a 3-D view of an object with the virtual sphere. http://www.panoramas.dk/US/golden-gate.html
12. The Bead Box sound should be used morein interface design, integrating music, voice, and more sophisticated sound effects. 70 percent of the population think they’re amateurmusicians, which is pretty high. age – type of music listened Gives opportunities to people who want to make music but have notmastered a conventional instrumentnotation or the meaning of a score.
13. Bill Gaver studied sound perception his work is about auditory icons meet Joy Mountford – started to work as an intern at Apple developed SonicFinder – tried to add sound to the software of Finder when you clicked on a file, you would hear a tapping noise
14. History Tablecloth The History Tablecloth uses weight sensing, measuring how longthings have been left on the surface. Provide us to see history of objects in the home load sensors calculate product’s position, and cells are lit to form a halo that grows slowly over a period of hours.
15. Key Table and Picture Frame provide a place for people to leave the things they carry with them when out of the house—keys, mobile phone contains a load sensor mounted under the top surface to measure the duration of oscillation caused by a new load, a good indication of impact force. When objects are put on the Key Table, this measurement is wirelessly transmitted to the picture frame, which tilts accordingly.
Notas do Editor
The entire floor and walls of a high-ceilinged516 | Chapter 8Room 3, Dreams—”The Realm of Simulation” •Arata Isozaki •rectangular hall were completely covered with moving images.Some of the time they were separated into individual imagesmaking a collage of screens, and some of the time they were hugepanoramas, stitched together to form a single image.“blurring the line between real and simulated”was demonstrated very convincingly by the exhibition, but theaudience were passive receivers of the experience, like cinemagoers;there were no interactive enhancements to their lives.
He tries to add the dimensions of physical feeling to the interface.
He tries to add the dimensions of physical feeling to the interface.
The opening and closing of a bottle is also detected. When a bottle is placed onto the stage area of the table and the cork is removed, the corresponding instrument becomes audible. A pattern of colored light is rear-projected onto the table's translucent surface to reflect changes in pitch and volume. The interface allows users to structure the experience of the musical composition by physically manipulating the different sound tracks
Durrell Bishop wants to design objects that are self-evident,whether they are physical or virtual. He looks at electronics andnotices that the shape of the objects does little to describe whatthey are actually doing; it is hard to see the difference between aradio and calculator by their design.
It has value; yes,it’s written on it what the value is, and its size and shape denotethat to some extent, but the real idea of value is not a physicalthing. It also has country; it belongs to a nation. But most of all,and strangest of all, it has ownership; the ownership isn’t representedon it at all, but we totally accept it, and it’s mostly defined by thedistance it is from you. If I’m holding it, it’s mine; if it’s in mypocket, it’s mine. If I place it far enough away, we start questioningwhose it is, but we don’t think about this as a pointer to money; it’snot a tag for money—it is money.
He started his ownproject without explaining to her what he was trying to do. Hehad a video camera pointing at an object, seemingly doing thesame thing day after day.
What really intrigued mewas how much variation of the audio or music mix you could actuallycreate, even with the simplest samples and changes. It was reallyquite addictive, and it became almost an art to see how wonderfulyour music could be with just three beads.
When objects are left on the table, load sensors under the table legs are used to calculate their position, and cells are lit to form a halo that grows slowly over a period of hours. When objects are removed, the corresponding halo is designed to disappear over a period of about 30 sec. The History Tablecloth makes visible the movement (or lack thereof) of objects in the home but leaves interpretation of that situation to the people who encounter the tablecloth.
