We present a novel manipulation method that subconsciously changes the walking direction of users via visual processing on a head mounted display (HMD). Unlike existing navigation systems that require users to recognize information and then follow directions as two separate, conscious processes, the proposed method guides users without them needing to pay attention to the information provided by the navigation system and also allows them to be graphically manipulated by controllers. In the proposed system, users perceive the real world by means of stereo images provided by a stereo camera and the HMD. Specifically, while walking, the navigation system provides users with real-time feedback by processing the images they have just perceived and giving them visual stimuli. This study examined two image-processing methods for manipulation of human's walking direction: moving stripe pattern and changing focal region. Experimental results indicate that the changing focal region method most effectively leads walkers as it changes their walking path by approximately 200 mm/m on average. Paper (PDF): http://akira.io/papers/om_paper_uist16.pdf Video: https://www.youtube.com/watch?v=RTGpr6p582o Project page: http://digitalnature.slis.tsukuba.ac.jp/2016/06/moh/

1. Optical Marionette:
Graphical Manipulation of Human’s Walking Direction
Akira Ishii, Ippei Suzuki, Shinji Sakamoto, Keita Kanai
Kazuki Takazawa, Hiraku Doi, Yoichi Ochiai
(Digital Nature Group, University of Tsukuba, Japan)

2. Background | Redirected Walking
n Manipulation technique of human’s walking direction
for VR environment
n Uses only visual feedback for the manipulation
n Enables users to explore a virtual world that
is quite larger than the real environment
2
Figure by
Matsumoto et al. 2016

3. Introduction | Motivation
n Suspected there are possibilities that human’s walking
direction could be manipulated using only visual feedback
in the real environment like a Redirected Walking in VR
3
To manipulate user’s walking direction
using only visual feedback in the real environment
Our goal

4. Introduction | To This End
n Examined various image-processing methods
to manipulate walking path
n Investigated how to manipulate the walking path
using the image-processing methods
4
We found effective reorienting method
using an HMD in the real environment

5. Implementation
n The camera is attached to the HMD
n Users perceive the real world by video
provided by an HMD and stereo camera
n So we can coordinate users’ sight in this setup
5
HMD Stereo camera
=
See-through

6. Implementation | Hardware
n HMD: Oculus Rift DK 2
n Stereo camera: Ovrvision
6
HMD
Camera

7. Implementation | Hardware
7
User’s sight via HMD + camera
while walking

8. Implementation | Flow
8

9. Implementation | Flow
9
Investigated effective
image processing in a pilot study

10. Pilot Study
n Purpose
- To determine which of the image processing methods
have the most effect on a human’s walking path
n Participants
- 5 participants (1 female); 18-22 ages
n 6 image processing methods
n Task
- Walk straight 10 m for each image processing methods
10

11. Pilot Study | Result
11

12. Implementation | Changing Focal Region (CFR)
12
Raw video
from camera
Video that users see
via HMD
Crops the raw image
Shifts the cropped area

13. Implementation | Changing Focal Region (CFR)
13
Raw video
from camera
Video that users see
via HMD
Crops the raw image
Shifts the cropped area
User is asked to go to the B, however, actually he is going to the A

14. Experiment
14
n Purpose
-To determine how to control the walking direction
using Changing focal region method
n Participants
-16 participants
-Had not prior knowledge
of our experiment
n Task
-Walk straight 24 m for each image processing

15. Experiment | Experimental Design
n Image processing methods
- No processing (raw video)
- Magnification
- Magnification was added to the image-processing methods for comparison
because it is used in Changing focal region method.
This enabled us to identify the effects of image magnification and
to reveal the pure effects of changing the focal region (shifting images).
- Changing focal region (slow / fast)
- Scroll speed of the cropped area in the slow condition was 0.5 px/frame
- In the fast condition, the scroll speed of the cropped area was twice
the scroll speed in the slow condition
15
Magnification + Shifting
Changing focal region

16. Experiment | Experimental Design
n Locations
-Hallway
- Narrow
- Several hints for spatial
perception (such as walls)
-Outside
- Large space
- No hint
for spatial perception
16

17. 1. Completed pre-SSQ (Simulator Sickness
Questionnaire)
2. Walked straight for 24 m each of image processing
at 2 locations
-At 4 m, guided to the left
-At 14 m, guided to the right
3. Completed post-SSQ
Experiment | Procedure
17

18. Result | Simulator Sickness Questionnaire
n Analyzed the SSQ scores with t-test
-No significant difference between pre-SSQ and post-SSQ
-Pre-SSQ: 4.7 (SD = 4.7)
-Post-SSQ: 6.7 (SD = 8.3)
18
I did not feel
any motion sickness J

19. Result | No processing
19
Position of participants [m]
Distance from starting point [m]

20. Result | Changing focal region (fast)
20
Position of participants [m]
Distance from starting point [m]

21. Result
21
No processing Changing focal region
No processing
Changing focal region

22. Result | Hallway
n Significant effect
-No processing ― Changing focal region (slow/fast)
-Magnification ― Changing focal region (slow/fast)
n No significant effect
-No processing ― Magnification
22
Position of participants [m]
No processing Magnification CFR (slow) CFR (fast)

23. Result | Hallway
n Amount of the manipulation for Changing focal region
-Slow: 65.3 mm/m
- If you walk 2 m, your position moves by 10 cm horizontally
-Fast: 75.3 mm/m
23
No
Amount of manipulation [mm]
* guide to left
** guide to right

24. Result | Outside
n Significant effect
-No processing ― Changing focal region (slow/fast)
-Magnification ― Changing focal region (slow/fast)
n No significant effect
-No processing ― Magnification
24
Position of participants [m]
No processing Magnification CFR (slow) CFR (fast)

25. Result | Outside
n Amount of the manipulation for Changing focal region
-Slow: 105.2 mm/m
-Fast: 199.2 mm/m
25
No
Amount of manipulation [mm]
* guide to left
** guide to right

26. Result | Pure Effect of CFR
n No significant effect
between No processing and Magnification
-This indicates that the cropped image by itself did not
affect the participants’ walking paths
-It is clear that the participants’ walking paths
were affected by movement of the cropped area of
Changing focal region
26

27. Result | Hallway vs. Outside
n Significant effect within the locations
-The outside participants were affected more
by the manipulation method
- In the hallway, the participants could not move more than 1.1 m
because the with of the hallway is 2.2 m
- Thus, the mean value of the position change in the hallway was
smaller than that in the outside
27

28. Result | Slow vs. Fast
n Changing focal region (fast) was significantly more
effective than the slow condition
-Slow: 105.2 mm/m
-Fast: 199.2 mm/m (at outside)
28

29. Discussion | Spatial Perception
n Past research reported Changing the FOV has effects
on spatial perception [1, 2]
n No horizontal spatial perception effect on our result
-because no significant difference
between No processing and Magnification
n By contrast, we could not determine
whether there was any depth perception effect
29
In this study
[1] Campos, J., Freitas, P., Turner, E., Wong, M., and Sun, H. The effect of optical magnification/minimization on distance estimation by stationary and
walking observers. Journal of Vision 7, 9 (June 2007), 1028.
[2] Kuhl, S. A., Thompson, W. B., and Creem-Regehr, S. H. HMD calibration and its effects on distance judgments. ACM Transactions on Applied
Perception 6, 3 (September 2009), 19:1–19:20.

30. Discussion | Cognitive Resource
n Conventional navigations require users to recognize
information (go to the right) and then follow directions
n Our method directly affects users’ bodies
so that it can control them
without requiring user recognition process
30
Does our method have lighter burden
than conventional navigations?

31. Discussion | Cognitive Resource
n Significant amount of cognitive resources is required
for redirected walking in VR [3]
Similarly, our method might require some cognitive
resources
31
We plan to investigate how much cognitive
resources are really required
by users to follow the manipulation
[3] Bruder, G., Lubas, P., and Steinicke, F. Cognitive resource demands of redirected walking. IEEE Transactions on Visualization and Computer
Graphics 21, 4 (April 2015), 539–544.

32. Discussion | Feasibility
n Demonstrated our method at SIGGRAPH 2016 E-Tech
n Over 700 people were successfully manipulated
32
Direction:
Go straight to B

33. Applications | AR Contents
n In see-through AR contents,
to control of human’s walking path is important
because users might conflict each other
33

34. Applications | Walker Navigation
n There is possibility of walker navigation system,
if we can increase
the amount of manipulation of users’ walking direction
34

35. Related Work | Galvanic vestibular stimulation (GVS) [4]
n Administers electrical stimulation to the back of ears
(the vestibules)
n Controls the walker’s sense of balance
35
[4] Maeda, T., Ando, H., Iizuka, H., Yonemura, T., Kondo, D., and Niwa, M. Parasitic humanoid: The wearable robotics as a behavioral assist interface
like oneness between horse and rider. In Proc. AH ’11, 18:1–18:8.

36. Related Work | Electrical Muscle Stimulation (EMS) [5]
n EMS-based walker navigation system
n Controls walker’s legs using EMS
36
[5] Max Pfeiffer, Tim Du ̈nte, Stefan Schneegass, Florian Alt, Michael Rohs. Cruise Control for Pedestrians: Controlling Walking Direction using
Electrical Muscle Stimulation. CHI 2015, pp.2505-2514, 2015.

37. Related Work
n These methods use electrical stimulation
n Our method is vision-based manipulation
technique of human’s walking direction using
only visual feedback
37
[4] Max Pfeiffer, Tim Du ̈nte, Stefan Schneegass, Florian Alt, Michael Rohs. Cruise Control for Pedestrians: Controlling Walking Direction using
Electrical Muscle Stimulation. CHI 2015, pp.2505-2514, 2015.

38. Optical Marionette:
n We found effective image-processing methods for
walker movement control with an HMD
n We investigated of the effects of
the image-processing methods via a user study
n Changing focal region method was most effective,
and changed walking path by about 200 mm/m
38
Akira Ishii, Ippei Suzuki, Shinji Sakamoto, Keita Kanai
Kazuki Takazawa, Hiraku Doi, Yoichi Ochiai (University of Tsukuba)
ishii@akira.io
Graphical Manipulation of Human’s Walking Direction