Do we really need game testers in development teams? What is it that defines the core competence of a tester, and does this competence add any value to the development team?

1. Do we really need Game Testers?
Do we really need game testers when we move to a more Agile way of
developing games? If the development teams own the quality of their product
and do their own testing, is there really a place for dedicated game testers? Play
tests with real users/untrained testers is one thing, but here I am talking about
professional testers performing testing throughout the game life cycle. To
answer this question we need to talk about competence instead of roles. In a
cross-functional development team, is there a specific competence that a game
tester can bring to the table, or can they just be replaced with more coders that
perform the testing themselves? I will try to answer this question based on my
own experience as a software tester, and introspection into how I approach
testing problems.
To find an answer, we need to break down test competence into its components,
and see if this competence is unique in any way, and if it adds value to the
development team.
Let us start this breakdown with trying to think about some general
competencies we expect game testers to have, but which are not unique to game
testers, and is something we expect from any developer or engineer. This will
obviously not be a comprehensive list, but let us try to cover many competencies
we usually associate with game testers, so that we can exclude them from the
elusive test competence, to get down to the core of what a game tester actually
brings to the table in a development team.
Competencies we (might) expect game testers (and most other people) to have,
which have nothing to do with test:
 Communication skills
 Collaboration skills
 Agile mindset
 Understanding business value
 Quick learner
 Critical thinker
 System modeling skills
 Coaching skills
 Understanding/writing code
 Reporting skills
 Risk analysis skills
 Review skills
 Being an active listener
 Being great at giving feedback
 Usability skills
 Domain knowledge
 Fine motor skills
 Reading comprehension

2. Even if you had all these skills (and you would be a great person if you did), there
is nothing in this list that specifically requires a tester. A great developer
probably has many of or all of these skills, and many additional development-
related skills.
So if test competence is not covered by the list above, what is it then? To
understand this, it is my belief that we must look in the Cynefin Framework [1].
There are many different types of problems, and according to this framework
they can be divided into four categories; Obvious, Complicated, Complex and
Chaotic.
It is my belief that higher levels of test competence are related to the ability to
handle complex test problems. So what does this mean? First, let us define a
complex test problem.
Obvious Test Problems: Tests in which the relationship between cause and effect is
obvious to all
Complicated Test Problems: Tests in which the relationship between cause and
effect requires analysis or some other form of investigation and/or the application
of expert knowledge
Complex Test Problems: Tests in which the relationship between cause and effect
can only be perceived in retrospect, but not in advance
So a game producer should have no problems solving an obvious test problem,
and a game developer should have no problems solving a complicated test
problem, but it is my belief that a game tester is better suited for solving complex
test problems due to their specific core competence.
Sidebar: How does complex
behavior arise?
“The behavior of a complex
system is often said to be due to
emergence.” [3]
“Emergence is a process whereby
larger entities, patterns, and
regularities arise through
interactions among smaller or
simpler entities that themselves
do not exhibit such properties.”
[4]

3. Let’s take a step back and look at the implications of what I just wrote. This
means that I see no reason to have dedicated game testers to handle obvious and
complicated testing problems. This can be done by developers, and in some cases
even producers, designers and other roles. Obvious and complicated testing
problems do not require high enough test competence to justify a dedicated
game tester on the team. Sometimes you might however need a tester to coach
the members of the team to raise their test competence to a level that allows
them to handle these obvious and complicated problems.
The next step is to define this test competence I keep mentioning. I believe it
consists of (at least) the following components:
 Exploring a complex system through tests
 Modeling unpredictability in a complex system
 Provoking a system to reach an unpredictable outcome
 Having a toolbox of unpredictable behavior
So what does this actually mean? What is it that I expect an experienced tester to
achieve with the four bullets above? How does this add value to the development
team?
If an experienced tester is confronted with a complex test problem, then this is
how I expect them to approach this problem:
 They start with a software system under test and some information about
this system (could be requirements, risks, historical data, etc.)
 Based on previous experience (either domain specific or general) and/or
based on a toolbox of testing heuristics [2], and the system information, I
expect them to come up with a number of test ideas
 Based on these ideas, they then model a faulty system (in their mind or
documented in the form of high-level test cases or test missions) which
could show an unpredictable behavior
 They then provoke the system to try to reach this faulty state (which may
or may not require specific tools, and/or understanding of the system)
o Documenting how to provoke the system to reach the faulty state
would probably result in a step-by-step detailed test case
 If they succeed they may have found a potential bug, and if the system
does not show unpredictable behavior then the system works according
to specification
 They then continue to explore the system using more test ideas, but also
equipped with the new information the previous test provided
By approaching a complex test problem like this, I believe that the game tester
will provide value to the development team by providing them with information
about how the system actually works, and by helping them understand and
disperse the complexity of the system.
So a game tester needs a wide array of skills to be effective (the first list of
competencies), but they also need a strong test competence, which is what

4. makes them unique in the development team, and allows them to provide value
in the form of a better understanding of the complexity of the system.
Everyone in a development team can test, but not everyone is equally equipped
to handle complex test problems.
So do we really need game testers? The answer is yes, and no. What we need is
test competence. If we can get this competence into the development team in
some other way, then no, we don’t need dedicated game testers. If we cannot,
then having game testers on the team may be the only solution.

5. References
[1] Cynefin Framework
https://en.wikipedia.org/wiki/Cynefin_Framework
[2] Heuristics Cheat Sheet
http://testobsessed.com/wp-
content/uploads/2011/04/testheuristicscheatsheetv1.pdf
[3] Complexity
https://en.wikipedia.org/wiki/Complexity
[4] Emergence
https://en.wikipedia.org/wiki/Emergence