Testing a game console isn’t all fun and games. However, with more than 50 million Xbox 360 consoles sold, and the amazing success of the Kinect sensor, it’s certainly a hotbed of excitement for software developers and testers alike. Veteran tester Alan Page is having a blast on the Xbox console team and shares an insider’s view of what it’s like to test one of the most popular entertainment systems ever created. Learn the details of testing the Xbox from the guts of the operating system to the latest applications—and everything in between. Discover how the Xbox team creatively balances test automation with hands-on testing, how they’ve turned traditional measures of code coverage and test pass rates on edge, and many other ways the Xbox team builds, tests, and delivers software that both end-users and game developers love. Join Alan to discover new ideas, see exciting demonstrations, and take away practical ideas that any team can use.

1. K2
Keynote
10/2/2013 10:00:00 AM
"Testing the Xbox: Lessons for
All"
Presented by:
Alan Page
Microsoft Corporation
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Alan Page
Microsoft
Alan Page is a principal SDET—a fancy name for tester—on the Xbox console team at
Microsoft. Edging up on twenty years in software testing, Alan has previously worked on a
variety of Microsoft products including Windows, Windows CE, Internet Explorer, and Office
Lync. He spent some time as Microsoft’s director of test excellence where he developed and ran
technical training programs for testers throughout the company.

3. 9/19/2013
Testing the Xbox
ALAN PAGE
MICROSOFT
Lessons Learned
from Xbox One
1

4. 9/19/2013
Lessons Learned
from Xbox One
My Story
2

5. 9/19/2013
October 31, 2011
3

6. 9/19/2013
Build a Great
Team
LESSON 1
Three Rules:
- Build a Team, Not a product
- Don’t Ship your Org Chart
- Leverage your Strengths
4

7. 9/19/2013
Have Super
Powers
LESSON 2
5

8. 9/19/2013
You Don’t Have to
Manage to Lead
LESSON 3
6

9. 9/19/2013
Have a Strategy
LESSON 4
7

10. 9/19/2013
Communicate
Intent and
Priorities
Have a plan for
Debt
LESSON 5
8

11. 9/19/2013
“The Tools, Systems, and
Culture we’re creating now…
…will be the Tools, Systems and
Culture we use for the lifetime
of the product.”
The Crowd is
Powerful
LESSON 6
9

12. 9/19/2013
The Crowd is
Powerful
…and the User is
King
LESSON 6 (REVISED)
Evaluation Scale
5
Love It! – You love it so much you’d shout it out at the top of your
lungs through a bullhorn from every rooftop you encounter
4
Like It – You like it a lot and might mention it during a lull in
conversation at a dinner party
3
Meh – You can live with it and neither like it or hate it. It’s nothing
special. There are some improvements that can be made
2
Don’t Like It – The experience leaves a bad taste in your mouth.
You’d use it if you really had to otherwise, you’d stay away from it.
1
Hate it! – You hate it so much that you would only use it if you
were under a Hogwarts compulsion spell
0
Not Implemented
10

13. 9/19/2013
Scenario
Love It Like It
Meh Don't Like Hate
Do this
43
10
1
1
1
Do that
3
2
9
14
20
Do the other thing
40
14
5
0
1
Do something else
20
3
5
7
15
11

14. 9/19/2013
OverCommunicate
LESSON 7
12

15. 9/19/2013
Have Fun
LESSON 8
13

16. 9/19/2013
http://angryweasel.com/blog
http://twitter.com/alanpage
14

17. 9/19/2013
Ease of Use
• Am I able to complete the scenario?
• Is it complicated? Do I need to perform
extraordinary steps to get what I need done?
• Are there glitches in the system that make it
difficult?
• Was it hard to find how to execute this
scenario? Are the features hidden?
• Is the experience consistent?
15

18. 9/19/2013
Responsiveness
• Did I feel like the scenario was fast and
fluid?
• At any point did I feel like I had to wait
a frustrating amount of time?
• Was I effectively distracted while
waiting for an action to take place? (ex.
Movie or animation while I wait)
Usefulness
• Would you use this scenario yourself
to accomplish this specific outcome?
• Does this scenario meet a need for our
consumer?
• Does this let me do something that I
want to do?
16

19. 9/19/2013
Visual Appeal
• Is the experience exciting to see
and hear?
• Is the UI polished?
• Does the UX make the experience
enjoyable?
17