Sergey Galyonkin, Head of Publishing, Epic Games

1. Understanding your game through data
Sergey Galyonkin

2. Understanding your game through data
2
Marketing analyst
Worked for Wargaming, Nival, 1C, handled Ukrainian distribution for
Activision Blizzard, EA, Ubisoft, Take Two and so on.
I also host a podcast about game development “How games are made”
and write a lot on Twitter as @Steam_Spy.
Sergey Galyonkin
Who am I?

3. Understanding your game through data
3
What I’m going to talk about
• There are many ways to obtain data that might
help you to improve your game.
• You should gather data on all stages of game’s
lifecycle, starting with pre-production and going
well into support and “$0.99 sales” stages.
• You should start to work with analytics before
you even launch the development.
• Not every tool mentioned will work for your
game, but most of them will.

4. Understanding your game through data
“Don’t tell me how to make my game”
“I’m a creator, not a corporate slave, and I don’t need
marketing research to make something truly unique.
Like retro-looking platformer for Steam.”

5. Understanding your game through data
5
Some arguments against doing your research
• “I’m a creator, I know what I’m doing and I don’t
want unwashed masses opinions on that”.
• You’re not asking for advice when you’re doing
research. You’re asking questions.
• “I’m making something truly unique, there is no
pre-existing market for it”.
• Awesome, but how do you know, if you didn’t do
a proper research first? Chances are there have
been games with mechanics similar to yours.
Don’t you want to learn on their mistakes?
• “I don’t care about the numbers, my goals are
qualitative, not quantitative”.
• Qualitative goals can be measured too. Don’t you
want to make sure you’ve achieved your targets?

6. Understanding your game through data
6
Some truisms
• You’re doing research and analytics to make
informed decisions about your game.
• You’re the one making decisions. Not the
audience, not Steam Spy, not Google Analytics.
• Data doesn’t “tell” anything, nor it is “good” or
“bad”. You’re the one interpreting it.
• There are many tools to do the research. Not
everything will be applicable to your game.
• One day spent doing the research could save a
month in development later.
• While doing research is an actual job (like
marketing or art), you can still do some of it
yourself without hiring other people.

7. Before the development
I don’t even have a game yet, why do I need to do
analytics at this stage?

8. Understanding your game through data
8
Why?
• It’s the best time to research the market, as you
are not invested in your game too deep yet –
both emotionally and financially.
• You can cancel or change the game after one
week of market research – it’s cheaper and easier
compared to years of development.
• You might improve your game even before you
start working on it.

9. Understanding your game through data
9
What do I need to research?
• The game’s theme
• The game’s core mechanics (genre)
• Audience
• Market
• Competition

10. Understanding your game through data
10
Theme, mechanics and visual style
• Your game could be described as intersection of
theme (sci-fi, fantasy, modern warfare), set of
core mechanics (FPS, RPG, side-scroller) and
visual style (80’s cartoon, surrealism, gritty, fake
retro).
• The theme is the easiest part to research

11. Understanding your game through data
11
Theme
• Look for movies, TV shows, comic books and books
sales and audience engagement stats for your
theme.
• Most of it could be obtained through Statista, Box
Office Mojo and IMDB. Or even Google Trends.
Steam Spy is not so great here.
• E.g. it’s obvious that the interest in sci-fi and space
opera is growing recently as we see more movies,
books and comic books in this theme.
• It’s hard to lead the trend - create a game that will
make everyone interested in sci-fi.
• It’s easier to follow - create a sci-fi game just in time
when people have an itch to fly space ships thanks
to a movie or book they’ve enjoyed recently.

12. Understanding your game through data
12
Theme

13. Understanding your game through data
13
Theme
• Predicting a trend is a bit harder, but it’s worth it.
Sci-fi growth was visible right after Avatar success
and we saw a number of games successfully
using it.
• Look for announcements of big movies with your
theme and chart them on a graph so you can see
when you can expect their marketing campaigns
to start (and help you).
• If your theme is too niche or too small – it’s fine
too. You’ll learn more when researching your
audience.

14. Understanding your game through data
14
Core mechanics
• Decompose your game into several core
mechanics that are easy to explain and preferably
match Steam’s tags. E.g. Turn-Based + Platformer
+ Clicker.
• Look for games that are using the same core
mechanics on Steam Spy or Google.
• Make a list of top games for each mechanics (or
better for their combination) and list their
strengths and weaknesses (gameplay-wise).
• You will later use this list to avoid making
mistakes other people did while developing
similar games and to improve your own game.

15. Understanding your game through data
15
Audience
• This one is tricky.
• Your potential audience are people with access
to the platform of your choice interested in your
game’s theme and its core mechanics and not
scared away by its visual style.
• You’ll probably have to use social media groups
or forums. I’d start with the ones dedicated to
your theme, but you can go with core mechanics
as well.
• Go, introduce yourself and just ask people if they
would be interested in playing something like
your game. Make a poll if the social media of
choice allows it.

16. Understanding your game through data
16
Caveats
• Don’t ask your friends and don’t promote that
poll on your own social media. You want an
opinion of your target audience.
• Use several independent forums or groups, not
just one.
• Don’t pay too much attention to game ideas
people will gladly provide you with. Not at this
stage at least.
• Be aware of sampling error (both oversampling
and undersampling). If, according to movies and
books with the same theme, your game’s theme
is more appealing to women over 40, don’t ask
twenty-something males about it.

17. Understanding your game through data
17
Competitors
• Make a list of games that are using the same
theme and some (or all) of your core mechanics.
Both released and upcoming.
• Make a basic SWOT analysis for all of them, but
also add an additional field: “Why our game is
different”.
• The key word here is “different” not “better”, so
you won’t get caught in wishful thinking “we’ll
have better graphics and better balance”.
• You can also check geographical distribution and
stats for released games on SteamSpy or
AppAnnie, but, frankly, it’s not that useful at this
stage.

18. Understanding your game through data
18
Summary
• A good analysis will take you a week or two, full-
time.
• It might seem like overkill, but it’s nothing
compared to doing the same during actual game
development and then changing the game
accordingly.
• Do it yourself, don’t outsource. You might come
up with new ideas for your game at this stage by
just researching relevant media and audience.

19. Understanding your game through data
During the development
Your choices now are becoming more and more
expensive.

20. Understanding your game through data
20
Why?
• It’s the time when you can get a reaction to your
game, not just an idea of it.
• You can change many things without angering
your existing audience.

21. Understanding your game through data
21
What do I need to research?
• An audience reaction to your visual style
• Players behavior in your prototype
• User interface

22. Understanding your game through data
22
Visual style
• If you’ve talked to people online about your
game in the previous stage, you should have a
pretty good understanding of who is the target
audience for your game.
• You can now ask them about your visual style.
Usually it’s enough to have several art
draft/mockups.
• Don’t ask if they like it or not, ask if they’d play a
game that looks like this.
• Focus on your target audience. Every successful
game is considered ugly by someone on Internet.
Skyrim? Too dirty. Clash of Clans? Too childish.
League of Legends? Too bright.

23. Understanding your game through data
23
Player behavior
• By now you should be adding in-game analytics
already. There are many options, you can choose
whatever you like.
• But even if it’s not there yet, you can learn a lot
by watching people play your prototype.
• The best way is to record a person’s face
simultaneously with what’s going on the screen
(think “letsplays”), but you could just sit nearby
and watch if the person is comfortable.
• Again, when inviting people to try your game,
focus on the target audience, not your friends.
• It will get harder in the end, as you’ll be running
out of relatives to test the game on.

24. Understanding your game through data
24
Player behavior
Things to look for when watching people play your
game:
• What catches their attention?
• What do they do first?
• How much time passes before the person gets to the
first major point in the game?
• Did you expect it to take this long?
• Do they understand the game rules?
• Do they understand effects of game items or enemy
strategies?
• How do they handle controls?
When a person starts playing, don’t talk, don’t help,
don’t defend or explain your game. Shut up and watch.

25. Understanding your game through data
25
Player behavior
• Always do a short interview and a survey after
the playtest. It helps to quantify and record the
results.
• Determine a net promoter score. Ask “On a scale
of 0 to 10, how likely are you to recommend this
game to a friend?”
• You can ask if the person liked the game or not,
but NPS is more useful, as people tend to lie so
they don’t hurt your feelings.
• Ask about what the person found most
confusing, most fun, most difficult and most
boring.

26. Understanding your game through data
26
Net Promoter Score
Test Detractors (0-6) Passives (7-8) Promoters (9-10) NPS Score
Idea test 1 (online poll on forum) 5 20 10 14%
Visual Style 1 5 7 6 6%
Visual Style 2 7 4 7 0%
Visual Style 3 7 6 5 -11%
Playtest 1 (studio) 5 7 6 6%
Playtest 2 (expo) 20 45 30 11%
Playtest 3 (online) 110 200 250 25%
How likely are you to recommend the game to your
friends? On a scale from 0 (not likely) to 10
(extremely likely)

27. Understanding your game through data
27
User Interface
• Prototype on paper. Get a notepad the size of
iPad or iPhone (they’re available online) or just
typical A4 one and draw stuff.
• If your game is for mobile devices, consider
different screen sizes and different hands sizes.
Test on different people.
• You’ll be able to eliminate many problems by just
handing the piece of paper to a person and
asking if he/she can understand how the game is
supposed to be played and what’s going on the
screen.
• I have a friend who builds elaborate paper UI
prototypes with moving parts, but it’s probably
too much. Quite fun to use, though.

28. Understanding your game through data
After the launch
The game is done, fire and forget.
Why are we still analyzing it?

29. Understanding your game through data
29
Why?
• You have actual users playing the game!
• You can still find and fix a lot of problems
• You’ll learn a lot for your next game

30. Understanding your game through data
30
What do I need to research?
• In-game behavior
• In-game economics
• In-game balance

31. Understanding your game through data
31
Behavior
• I hope that by release you have some analytics
system implemented – Google Analytics, Unity
Analytics or even home-grown solution will do.
• You should be tracking user path through the
game.
• Cluster users based on the steps they’re taking in
your game. Some will go straight to tutorial,
some will go PVP, some will start exploring.
• How much time a person spends in a game per
sitting? How often does he play per week?
• Don’t go for averages, always go for brackets or
clusters, as averages are deceiving.

32. Understanding your game through data
32
Economy
• How many resources a person has after 1 hour /
1 chapter into the game? How many people have
too much and how many don’t have enough?
• What people usually spend their resources on?
Do they go for armor, swords, potions, artillery or
tanks? Why?
• Cross-reference the economy with behavior
clustering. Do “explorers” buy different items
compared to “fighters”? Do they have more
resources or less? Should it be like this?
• Research consumables (potions, gems, etc). Do
people actually use them or save them for later?

33. Understanding your game through data
33
Economy
Expected valueDidn’t
understand the
game at all
Low skill, will
have problems
later in the
game
High skill, might
find the game
too easy later
Munchkins?
Exploiters?

34. Understanding your game through data
34
Balance
• Which character/class/role people choose more
often? Does it vary based on their behavior in
game?
• Based on the character selected are there any
differences in resources acquired, damage dealt,
time spent in game?
• If the game has PVP, check the win/lose ratio for
every class. Again, use brackets or clustering
instead of averages, as win ratio might vary a lot
depending on users’ skill.

35. Understanding your game through data
35
Balance
The 1st chapter
Class Warrior Paladin Priest Wizard Monk
Total players 600 200 100 50 50
Share of all players 60% 20% 10% 5% 5%
Resources after 252 170 150 405 180
DPS after 45.2 35.2 17.2 92.5 40.2
Average time to beat 01:05 01:30 02:30 00:55 01:15
The 2nd chapter
Class Warrior Paladin Priest Wizard Monk
Total players 550 180 70 47 47
Share of all players 62% 20% 8% 5% 5%
Resources after 630 442 390 810 450
DPS after 90.4 70.4 43 148 80.4
Average time to beat 01:02 01:20 03:00 00:50 01:10
The 3rd chapter
Class Warrior Paladin Priest Wizard Monk
Total players 545 175 35 45 45
Share of all players 64% 21% 4% 5% 5%
Resources after 1575 1105 975 1782 1125
DPS after 180.8 140.8 107.5 236.8 160.8
Average time to beat 01:07 01:25 03:30 00:40 01:15
What can we learn here?
• Priest class might be severely
underpowered. Players are
leaving the game.
• Wizard class is either too
strong or more appealing to
more skilled players
• Monk and Paladin are all right
• Warriors might need some
balancing, so they spend more
resources

36. Understanding your game through data
Summary
A quick recap

37. Understanding your game through data
37
Key takeaway
Do your research
before developing a game.
You’ll both save time
and learn a lot.

38. Understanding your game through data
38
Key takeaway
Test your game
on your target audience
even before you have
anything playable.

39. Understanding your game through data
39
Key takeaway
Implement in-game analytics
as early as you can,
it makes analyzing
user behavior way easier.

40. Understanding your game through data
40
Key takeaway
Don’t trust averages,
look at the whole picture.

41. Understanding your game through data
41
Key takeaway
Don’t dismiss analytics
as something
for “the big guys only”.
You can do a lot of it yourself
or with a small team.

42. Understanding your game through data
42
Questions?
Skype
sergesegal
E-mail
sergey@galyonkin.com
Twitter
@Steam_Spy
What?
How?
When?
Why?
Where?
Are we
going to
die or
not?
Sergey Galyonkin