The thin-client model is considered a perfect fit for online gaming. As modern games normally require tremendous computing and rendering power at the game client, deploying games with such models can transfer the burden of hardware upgrades from players to game operators. As a result, there are a variety of solutions proposed for thin-client gaming today. However, little is known about the performance of such thinclient systems in different scenarios, and there is no systematic means yet to conduct such analysis. In this paper, we propose a methodology for quantifying the performance of thin-clients on gaming, even for thin-clients which are close-sourced. Taking a classic game, Ms. Pac-Man, and three popular thin-clients, LogMeIn, TeamViewer, and UltraVNC, as examples, we perform a demonstration study and determine that 1) display frame rate and frame distortion are both critical to gaming; and 2) different thin-client implementations may have very different levels of robustness against network impairments. Generally, LogMeIn performs best when network conditions are reasonably good, while TeamViewer and UltraVNC are the better choices under certain network conditions.

1. Understanding the Performance of
Thin-Client Gaming
12011/5/11 CQR 2011 / Yu-Chun Chang
Yu-Chun Chang1, Po-Han Tseng2, Kuan-Ta Chen2, and Chin-Laung Lei1
1Department of Electrical Engineering, National Taiwan University
2Institute of Information Science, Academia Sinica

2. Outline
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11 2CQR 2011 / Yu-Chun Chang

3. Introduction (1/2)
3
Client Server
User’s inputs
Display updates
• Thin-client system
2011/5/11 CQR 2011 / Yu-Chun Chang

4. Introduction (2/2)
• Motivation
– To understand which performance metric is more
sufficient for thin-client gaming
• Frame rate, frame delay, frame loss, and etc
• Challenges
– Most thin-client products are proprietary
• Image compression, data-transmission protocol and display update
mechanism
42011/5/11 CQR 2011 / Yu-Chun Chang

5. Our focus
52011/5/11
QoE
Perf.
Metric
Network
Condition
Server
Client
Thin-client program
User
Network
Condition
Server
CQR 2011 / Yu-Chun Chang
QoE
Perf.
Metric

6. Outlines
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11 6CQR 2011 / Yu-Chun Chang

7. Experiment Methodology
72011/5/11

8. Why Do We Use Ms. Pac-Man?
• Move Pac-Man to eat pills and get the score
• Control through thin-client applications and move
Pac-Man in the game of server
– Good network condition: score↑
– Bad network condition: score↓
• Score  Quality of Experience
2011/5/11 CQR 2011 / Yu-Chun Chang 8

9. Ms. Pac-Man & Bot
9
• Ms. Pac-Man
– Save score after the pacman ran out of 3 lives
• Bot: ICE Pambush3 (published in IEEE CIG 2009)
– Java-based controller to move the pacman
– Capture the screen of the game and determine the position of the
pacman, ghosts, and pills
Number Score
Pill 220 10
Power pill 4 50
Ghost 4 200 (after eating power pills)
2011/5/11 CQR 2011 / Yu-Chun Chang

10. • Three thin-client systems
– LogMeIn
– UltraVNC
– TeamViewer
• Network conditions
10
Network condition Settings
Network delay 0 ms, 100 ms, 200 ms
Network loss rate 0%, 2.5%, 5%
Bandwidth Unlimited, 600 kbps, 300 kbps
2011/5/11 CQR 2011 / Yu-Chun Chang

11. • Performance metric
– Display frame rate
– Frame distortion (MSE: Mean Square Error)
• Record game play as video files in 200 FPS
112011/5/11 CQR 2011 / Yu-Chun Chang

12. Outlines
• Introduction
• Experiment methodology
– Experiment setup
– Performance metric extraction
• Performance evaluation
• Conclusion & future work
2011/5/11 12CQR 2011 / Yu-Chun Chang

13. 132011/5/11
Thin Clients are Different!
CQR 2011 / Yu-Chun Chang

14. Visual Difference Really Matters!
142011/5/11 CQR 2011 / Yu-Chun Chang

15. Statistical Regression
15
Regression
Model
QoE
(score)
Independent factors
Display frame rate
Frame distortion
2011/5/11 CQR 2011 / Yu-Chun Chang

16. Frame-Based QoE Model
• Linear model
• QoE =
16
Adjusted R-squared: 0.72
2011/5/11 CQR 2011 / Yu-Chun Chang

17. Frame-Based QoE Model
17

18. Which Performance Metric is More Sufficient?
• QoE degradation
– Optimal user’s QoE – user’s QoE predicted by model
• Frame rate is
more sufficient!
2011/5/11 18

19. Frame Rate and Network Conditions
19
Network
Condition
2011/5/11
QoE
Perf.
Metric
Server
Client
Thin-client program
User
CQR 2011 / Yu-Chun Chang

20. The Frame Rate Prediction Model
• Frame rate =
• app1, app2: dummy variables
– LogMeIn : app1 = 1, app2 = 0
– TeamViewer : app1 = 0, app2 = 1
– UltraVNC : app1 = 0, app2 = 0
• d: delay, l: loss rate, b: bandwidth
• dl, dt, du : delay of LogMeIn, delay of TeamViewer, delay of UltraVNC
2011/5/11 20CQR 2011 / Yu-Chun Chang

21. The Frame Rate Prediction Model
2011/5/11 21
Adjusted R-squared: 0.85
CQR 2011 / Yu-Chun Chang
Delay of LogMeIn
Delay of UltraVNC
Bandwidth of LogMeIn
Bandwidth of UltraVNC

22. Predicted Frame Rate
2011/5/11 22CQR 2011 / Yu-Chun Chang
Network delay Bandwidth

23. Which Thin-Client is Better?
23
Network
Conditions
2011/5/11
QoE
Perf.
Metric
Server
Client
Thin-client program
User
CQR 2011 / Yu-Chun Chang

24. Network-Based QoE Model
• QoE =
2011/5/11
Adjusted R-squared: 0.81

25. The Thin-Client with Best Performance
• o symbol: empirical network condition
– 300 records collected by PingER project
2011/5/11 25CQR 2011 / Yu-Chun Chang

26. Conclusions & Future Work
• Display frame rate and frame distortion are both
critical to gaming performance on thin-clients
• LogMeIn performs the best among the three
implementations we studied
• Future work
– Add more thin-clients to compare the performance
– Design a generalizable experiment methodology for thin-
client gaming with different game genres
2011/5/11 26CQR 2011 / Yu-Chun Chang

27. Thank you for your attention!
2011/5/11 CQR 2011 / Yu-Chun Chang 27