1. Speaker: Yi-Ping Hung （洪一平）
Professor and Director
Graduate Institute of Networking and Multimedia
National Taiwan University
Joint Research Fellow
Institute of Information Science
Academia Sinica

2. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
2

3. INTERACTION
between Human and Machine
Input Method Output Method
Card Punch Machine Line Printer
keyboard CRT Screen
Mouse LCD Screen
Finger Touch Touch Screen
 Other interaction methods, such as speech,
gesture, bio-signals, etc.
3

4. Touch-Control Handheld Devices
 iPhone  iPad
Apple Computer, 2007 Apple Computer, 2010

6. Hand Gesture Recognition
 Microsoft Kinect  SixthSense
controller-free gaming and entertainment WUW – Wear Ur World – A Waerable
experience for the Xbox 360, E3 2009. Gesture Interface, SIGCHI 2009
6
http://www.youtube.com/watch?v=qC3H3JOt
vSs

7. Intangible Displays
 FogScreen  Heliodisplay
http://www.fogscreen.com http://www.heliodisplay.eu/tech.htm

8. Related Work
─ Bio-Signal, Brainwave
8

9. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
9

10. Digitization of 2D Museum Collections
National Palace Museum
 Chinese Paintings
 Chinese Calligraphies
10

11. Digitization of 2D Museum Collections
 Reprographic Scanner  GigaPixel Camera
11

12. Digitization of 3D Museum Artifacts
Historical Museum of Academia Sinica
 3D Collection of Shoung dynasty
National Museum of History, Taipei
 Tricolors of Tang dynasty
12

13. Digitization of 3D Museum Artifacts
National Palace Museum
 5 artifacts selected for virtual exhibition
13

14. Object Movies
 Image-Based Rendering
 Equipments for images acquisition
 Texnai, Nodal Ninja, PeaceRiver Studios

14

15. Acquisition of Object Movies
Object-centric representation
CO
3D configuration of views
15

16. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
16

17. Integration of Object Movies with Panoramas
 Augmented Reality Based
on Cuboid Structure
○ Chen, Yu, Hung, ICCV’99
 Augment a Panorama
with Object Movies
 Hung, et al, JVCA 2002
Yi-Ping Hung, et al, “Augmenting Panoramas with Object Movies ……,”
Journal of Visualization and Computer Animation (JVCA), 2002. 17

18. Interactive 3D Virtual Exhibition
 Design a Stereoscopic Kiosk for Virtual Exhibition
3D Virtual
Exhibition
Viewer
Sync.
Stereo
Object Movies
18

19. Stereoscopic Kiosk
• Contains two screens
─ the upper one with touch panel
─ the lower one with micropol for
stereoscopic view
19

20. Stereoscopic Kiosk
 F@imp 2004 (Festival Audiovisuel International
Musees et Patrimoine)
20
20

21.  Daily Exhibition at the
National Palace Museum
2004/10 – 2005/10
21

22. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
22

23. Magic Crystal Ball（魔幻水晶球）
─ Magical Visual Experience
─ Intuitive Manipulation
• VRST 2007

24. System Archetecture
Transparent Glass Ball
Fresnel Lens
Touch Screen
Reflection Mirror
Two IR Cameras
IR Illuminators
Strain Gauge Sensors
Cantilever beam structure
LCD
Display Detection
Module Module

25. Hand Gesture Recognition
 Cover the spherical ball with two cameras
Side view
With one camera With two cameras

27. IRON MAN
27 /65

29. Virtual Panel
2D Planar Intangible Screen
LCD Screen
Virtual Panel
Fresnel Lens
IR Cameras

30. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
30

31. Interactive Multi-resolution TableTop Display（i-m-Top）
─Rear Projection
─Multi-Touch
─Multi-Resolution
• “i-m-Top: An Interactive Multi-Resolution Tabletop System Accommodating To Multi-Resolution Human Vision”, IEEE Tabletops and
Interactive Surfaces, Amsterdam, IEEE Tabletop 2008.
• To Move or Not to Move: A Comparison between Steerable and Fixed Regions of High-Resolution Projection in Multi-Resolution
Tabletop Systems, ACM SIGCHI 2009 31

32. Motivation for Multi-resolution
 Human Visual System: Multi-resolution
 Fovea region: acute visual details
 Peripheral region: coarse visual details

33. Related Works
 Focus + Context
Baudisch, P., Good, N., and Stewart, P. "Focus Plus Context Screens: Combining Display Technology with Visualization
Techniques." In proceedings of ACM UIST '01, Orlando, FL, November 2002

34. Related Works
 Escritoire
 A multi-resolution tabletop display system with front-projection
 Interact by two pens
 Fovea area is fixed
Mark Ashdown, Peter Robinson, “Experiences Implementing and Using Personal Projected Displays,” University of Cambridge
Procams’03 "

35. i-m-Top System
96 ppi
24 ppi
56”
Steerable Mirror
Fovea Projector
IR Camera x 2 Peripheral Projector
IR Illuminator x4

36. Some Applications
 Museum Painting Browser
 Map Browser
 Photo Browser

37. Map Touring with Tangible Widget
LCD Screen (Panorama view)
Tangible object (You)
im-Top (map view)

38. Tangible Widgets
for Virtual Exhibition
of Museum Collections
38

39. Yongzheng Exhibition
-- National Palace Museum
-- Oct 2009 ~ Jan 2010
39

40. Beyond-Surface Interaction
with Infrared Projection
Three Applications
40

41. IDEA: Using an Invisible Projection
i-m-Lamp
Location & Orientation
-- 6 DOF
i-m-Flashlight
Color
Projector IR Projector
i-m-View
Mirror
Mirror
IR Camera
41

42. i-m-Lamp
 Project high-resolution content on the
surface, mimicking the use of a desk lamp.
42

43. i-m-Flashlight
 A mobile version of i-m-Lamp, facilitating
information explorations and cooperative
tasks.
43

44. i-m-View
 An intuitive tool to view 3D content or
augmented information from different
perspectives.
44

45.  Demo
 Emerging Tech in SIGGRAPH 2010
 Publication
 UIST 2010
 Press Coverages:
 New Scientist
 Engadget
 GearLog
 Reuters
 International and domestic TV
45

46. OUTLINE
 Introduction
 Digitization of Museum Collections
 Interactive Systems for Virtual Exhibition
 Stereoscopic Kiosk
 Magic Crystal Ball
 Interactive Multi-Resolution Tabletop Displays
 Turning Rust into Gold
46

47. Turning Rust into Gold
 The Mao-Kung Ting
 One of the most precious artifacts in
the National Palace Museum, Taipei
 Dating back to 800 B.C.
 Having 500-character
inscription cast inside
 How to reveal the great nature
of this artwork?
 How to set up the connection
between museum visitors and this artwork?

48. Why Breath-Aware Sensor?
 The Power of the Breath
 Breathing is a function in our bodies that we
can do both consciously and unconsciously
 Breathing is a bridge between mind and body
 Yoga, meditation, Qi-Gong, Tai-Ji, … all focus
on breathing
 Be fully conscious of each breath
○ make you healthier and enlightened
○ enlightened: feel oneness with Being

49. Respiration Detectors
 Which Method for Respiration Detection ?
 Optoelectronic Plethysmography (OEP)
 Respiratory Inductive Plethysmography (RIP)
 Heart Rate Variability (HRV)
 Ultra Wide Band (UWB)

50. Respiration Detection
Comparison of Different Respiration Detection Methods
OEP RIP HRV UWB
Attached Attached Attached Attached Non-
to body markers to to invasive
Body skin
Accuracy High Median Median Low
Least Detectors Detectors Less
Limitation mobility attached attached mobility
to to
the body fingers

51. Turning Rust into Gold
點蝕成金
 K. T. Creativity Award 2009: is now demonstrated in the NPM LOHAS Exhibit
at the Future Museum, located at the Taiwan Taoyuan International Airport.

52. National Palace Museum
-- Future Museum
 Location: Taiwan Taoyuan International Airport
 LOHAS Exhibition, April/2010 –April/2011

56. Thanks