AI is used to create parts of our games. It provides intelligent enemy behavior, techniques such as pathfinding or can be used to generate in-game content procedurally. AI can also play our games. The idea to train computers to beat humans in game-like environments such as Jeopardy!, Chess, or soccer is not a new one. But can AI also design our games? The role of Artificial Intelligence in the game development process is constantly expanding. In this talk, Dr. Pirker will talk about the importance of AI in the past, the present, and especially the future of game development.

1. S C I E N C E * PA S S I O N * T E C H N O L O G Y
WHY AI IS SHAPING OUR GAMES
D R . J O H A N N A P I R K E R , T U G R A Z , A U S T R I A
K L A G E N F U RT 2 0 1 9

3. AI MIRACLES..

4. “MAKING COMPUTERS ACT LIKE THEY DO IN THE MOVIES.”

5. 1. THE CAPABILITY OF A MACHINE TO IMITATE
INTELLIGENT HUMAN BEHAVIOR.
2. A BRANCH OF COMPUTER SCIENCE DEALING WITH THE
SIMULATION OF INTELLIGENT BEHAVIOR IN COMPUTERS.
Merriam-Webster defines artificial intelligence this way.

6. “REAL” AI
▸ 1. learn over time in response to changes in its
environments
▸ (e.g. Netflix recommendations but not Twitter
black lists)
▸ 2. what it learns should be interesting enough that it
would take humans some effort to learn
▸ (Turing test)

8. AI IN GAMES
▸ … generate responsive, adaptive, & intelligent behaviour
▸ uses path finding, decision trees, data mining, PCG, …
▸ usually do not facilitate computer learning
▸ -> predetermined & limited set of responses to a limited set of inputs
▸ ILLUSION OF INTELLIGENCE
▸ good gameplay without environment restrictions
▸ learn & use from “real AI” strategies
▸ Learning Tamagotchi

9. ▸ decision trees (scripting)
▸ -> AI stupidity, predictive behaviour, loss of immersion
▸ pathfinding
▸ (Half Life, “Crouch Cover”)
▸ NPC behaviour in Doom
▸ NPCs fighting NPCs
AI IN GAMES - ISSUES

10. PLAY
GAMES.
CONTRIBUTE
CONTENT.
DESIGN
GAMES.
UNDERSTAND
PLAYERS.

11. I. PLAY GAMES.

12. AI TO PLAY GAMES
ROBOCUP

13. AI TO PLAY GAMES
CHESS - IBM DEEP BLUE VS. GARRY KASPAROV (1997)
"I could feel — I could smell — a new kind of intelligence across the table,"

14. AI TO PLAY GAMES
JEOPARDY! - IBM WATSON VS. KEN JENNINGS (2011)
"I could feel — I could smell — a new kind of intelligence across the table,"

15. AI TO PLAY GAMES
GO - GOOGLE ALPHAGO (DEEPMIND) VS. LEE SEDOL (2016)

16. AI TO PLAY GAMES
DEEPMIND VS. STARCRAFT II (2019)

17. AI TO PLAY GAMES
http://gameaibook.org/book.pdf

18. ▸ Chess Two-player adversarial, deterministic, fully observable,
branching factor ~35, ~70 turns
▸ Go Two-player adversarial, deterministic, fully observable, branching
factor ~350, ~150 turns
▸ Frogger (Atari 2600) 1 player, deterministic, fully observable, bf 6,
hundreds of ticks
▸ Halo 1.5 player, deterministic, partially observable, bf ???, tens of
thousands of ticks
▸ Starcraft 2-4 players, stochastic, partially observable, bf > a million,
tens of thousands of ticks
▸ Togelius
AI TO PLAY GAMES

19. AI TO PLAY GAMES
TRAIN AI HOW TO PLAY SNAKE (DEEP REINFORCEMENT LEARNING)
On the left, the agent was not trained and had no clues on what to do whatsoever. The game on the right
refers to the game after 100 iterations (about 5 minutes). The highest score was 83 points, after 200
iterations.
https://github.com/maurock/snake-ga

20. AI TO PLAY GAMES
TRAIN AI HOW TO PLAY STARCRAFT
‣ A Machine Learning API developed by Blizzard that gives researchers and developers hooks into the game.
‣ A dataset of half a million anonymised game replays,.
‣ An open source version of DeepMind’s toolset, PySC2
‣ A series of simple RL mini-games to test the performance of agents on specific tasks.
https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/

21. AI TO PLAY GAMES
WHY USE AI TO PLAY GAMES?
▸ Playing to win vs playing for experience
▸ For experience: human-like, fun, predictable…?
▸ Playing in the player role vs playing in a non-player role
http://gameaibook.org/book.pdf

22. METHODS
▸ Planning-Based
▸ Uninformed search (e.g. BFS),Informed search (e.g. A*),
Evolutionary algorithms
▸ Reinforcement learning (training time)
▸ TD-learning / approximate dynamic programming,
Evolutionary algorithms
▸ Supervised learning (requires play traces to learn from)
▸ Neural nets, k-nearest neighbors etc
▸ Random (requires nothing)
AI TO PLAY GAMES
▸ Togelius

23. II. CONTRIBUTE CONTENT.

24. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

25. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

26. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION

27. CONTRIBUTE CONTENT
PROCEDURAL CONTENT GENERATION
• Artistic aspects
• Corner-cases
• Lack of complete control
• Depends on the content
• Client-side calculations?
• Replayable content?
• Cheap
• Lots of content
• Dynamic Reaction on player
• Reduce burden of artist
• Save memory
• Large worlds
• Replayable content
• http://pcg.wikidot.com/category-pcg-algorithms

28. METHODS
▸ Search-Based Methods
▸ Solver-Based Methods
▸ Grammar-Based Methods
▸ Cellular Automata
▸ Noise and Fractals
▸ Machine Learning
CONTRIBUTE CONTENT

29. GENERATE CONTENT FOR…
▸ Environments (Random Maps, Random Dungeons)
▸ Generative Art and models
▸ Textures
▸ Music
▸ Story
▸ Gameplay
CONTRIBUTE CONTENT

30. III. UNDERSTAND PLAYERS

31. PLAYER MODELING
▸ … detection, prediction and expression of human player
characteristics that are manifested through cognitive,
affective and behavioral patterns while playing games
▸ can be used to dynamically adjust the gameplay (dynamic
difficult adjustment)

32. BEHAVIOURAL PROFILING

33. B A R T L E ’ S G A M E R T Y P E S
http://www.gamerdna.com/quizzes/bartle-test-of-gamer-psychology

34. Story
Story Enjoyer
Party Player
Killer
Online Hero
Allrounder
0% 20% 40% 60% 80% 100%
Story Enjoyer
Party Player
Killer
Online Hero
Allrounder
Time spent
Story
Campaign
Arena
Online MulAplayer
Local MulAplayer
P L AY E R H A B I T ( P L AY E R F I N G E R P R I N T )

35. P L AY E R P R O F I L E S I N F O R Z A
• What Drives People: Creating Engagement Profiles of
Players from Game Log Data
• 120 mio race entries from 1.2 mil players
•
Harpstead, E., Zimmermann, T., Nagapan, N., Guajardo, J. J., Cooper, R., Solberg, T., & Greenawalt, D. (2015, October). What Drives People: Creating Engagement Profiles of Players from Game Log Data. In Proceedings of the
2015 Annual Symposium on Computer-Human Interaction in Play (pp. 369-379). ACM.

36. F L O W ( M I H A LY C S I K S Z E N T M I H A LY I )

37. HOW PLAYSTYLES EVOLVE:
PROGRESSION ANALYSIS AND
PROFILING IN JUST CAUSE 2
https://link.springer.com/chapter/10.1007/978-3-319-46100-7_8

38. D ATA S E T
• Dataset provided by Square Enix
• Play histories from over 5000 JC2 players (2010)
• Various behavioural features collected:
• actions with
• in-game geographical coordinates
• timestamps
• metrics from the gameplay
• e.g. total kills, total chaos, kilometres driven # of stronghold
takeovers ,…
• Data set pre-processing (cleaning):
• Outliers removed: scores outside 1-99th percentile excluded
• (faulty tracking or errors)

39. F E AT U R E S
• Agency missions (+ reach specific level of Chaos)
• subset of features based on the core mechanics
• -> does not impact the analytical framework
• -> impacts the kinds of conclusions that can be
derived

40. F E AT U R E S
• Spatio-temporal navigation
• combat performance
• progression through the main storyline
• side quests..
• Agency missions (+ reach specific level of Chaos)
• subset of features based on the core mechanics
• -> does not impact the analytical framework
• -> impacts the kinds of conclusions that can be derived

41. P L AY E R P R O G R E S S I O N A L O N G T H E
M I S S I O N S

42. R E S U LT S
• How can we describe player behaviour of the
different player profiles?

43. P L AY E R B E H AV I O U R A L O N G T H E
S T O RY L I N E
jpirker.com/jc2/aaSankey.html

44. S O C I A L N E T W O R K S I N D E S T I N Y
Rattinger, A., Wallner, G., Drachen, A., Pirker, J., & Sifa, R. (2016, September) Integrating and Inspecting Combined Behavioral Profiling and Social Network Models in Destiny,15th International Conference on Entertainment
Computing (in press).

45. NETWORK RELATIONSHIP
‣ Jammer Network
‣ three-year span
‣ v: jammers
‣ e: developed a game
together
‣ undirected, weighted graph
‣ (weight: # games developed
together)
JAMMER 1
JAMMER 2
JAMMER 3
3
1

46. NETWORK

47. NETWORK

48. G O A L S
• Improve our understanding of the different player
behaviours and factors to improve engagement
• Find issues to avoid drop-outs
• Provide tools for game designers to (visually)
analyse the game and improve the understanding
of players
• Find game design flaws early and automatically

49. IV. DESIGN GAMES

50. AI AS A PART OF
GAME DESIGN!!!!

51. AI TO DESIGN GAMES
ROLES OF AI IN GAMES
▸ AI in the foreground of games - Foregrounding AI
▸ create gameplay based around thinking about how agents
work
▸ Designing games that use AI techniques in a new way as a
core of their gameplay
https://medium.com/@mtrc/tombs-of-tomeria-7c2e800a6511
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.

52. AI-BASED GAME DESIGN
▸ Game design strategies/rules described when AI still
“young” and most games are designed to not need AI
▸ Game designers often claim that AI won’t make games
better
▸ Our goal: show where AI can be used, show alternative
routes
▸ we need to design new games from scratch based on
new design principles
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.
AI TO DESIGN GAMES

53. AI GAME DESIGN PATTERNS
Mike Treanor, Alexander Zook, Mirjam P Eladhari, Julian Togelius, Gillian Smith, Michael Cook, Tommy Thompson, Brian
Magerko, John Levine and Adam Smith: AI-Based Game Design Patterns. Computational Creativity and Games Workshop,
2015.
AI TO DESIGN GAMES

54. AI DESIGN PATTERNS
1 AI IS VISUALIZED
▸ Pattern: Provide a visual representation of the underlying AI state, making gameplay revolve around
explicit manipulation of the AI state.
▸ Example: Third Eye Crime is a stealth game that illustrates this pattern by visualizing the guard AI position
tracking and estimation system. Gameplay involves avoiding guards or throwing distractions to manipulate
the guards’ predictions of player location. The direct visualization of AI state allows a designer to build a
game around manipulating, understanding, and mentally modeling how the AI state changes.

55. 2 AI AS ROLE-MODEL
▸ Pattern: Provide one or more AI agents for the player to behave similarly to.
▸ Example: Spy Party is a game where one player is a spy at a party populated by FSM agents and the
opposing player is a sniper watching the party with a single shot to kill the spy. Gameplay for the
spy centers on the player attempting to act similarly to the party agents while discreetly performing
tasks in the environment like planting a bug or reading a code from a book.
AI DESIGN PATTERNS

56. 3 AI AS TRAINEE
▸ Pattern: Have player actions train an AI agent to perform tasks central to gameplay.
▸ Example: Black & White is a god game where the player trains a creature to act as
an autonomous assistant in spatial regions where the player cannot take direct
action. The creature learns sets of behaviors through a reward signal based on a
needs model; the creature also takes direct feedback through player action (e.g.,
slapping or petting the creature after it takes actions).
AI DESIGN PATTERNS

57. 4 AI IS EDITABLE
▸ Pattern: Have the player directly change elements of an AI agent that is central to gameplay.
▸ Example: Galactic Arms Race is a space shooter where how the player uses different weapons evolves an underlying neural
network representation to change weapon firing behavior. Base gameplay revolves around finding a set of firing behaviors that
together enable a player to succeed at destroying opposition (another example of the AI as Trainee pattern). One gameplay mode
allows the player to explicitly manipulate the network weights on weapons, allowing more precise control over the firing patterns
of the evolved weapons. This control enables players to more finely explore the space of parameterizations, leading to an indirect
way to understand the processes of the AI system.
Erin J. Hastings, Ratan K. Guha, and Kenneth O. Stanley (2009)
Automatic Content Generation in the Galactic Arms Race Video Game
In: IEEE Transactions on Computational Intelligence and AI in Games, volume 1, number 4, pages 245-263, New York: IEEE Press, 2009. (Manuscript 19 pages)
AI DESIGN PATTERNS

58. 5 AI IS GUIDED
▸ Pattern: The player assists a simple or brittle AI agent that is threatened with self-destruction.
▸ Example: The Sims addressed the problem of “human-like” agents in a social world by making
gameplay revolve around the player addressing the needs of simple agents. AI agents have a set of
needs and desires they attempt to pursue while players intervene to provide for the needs of the
agents through food, shelter, work, socialization, and eventually more grand life aspirations. By having
players care for the AI, players come to (at least indirectly) model some of the processes used by the AI.
AI DESIGN PATTERNS

59. 8 AI AS VILLAIN
▸ Pattern: Require players to complete a task or overcome an AI opponent where the AI is aiming to create an
experience (e.g., tension or excitement) rather than defeat the player.
▸ Example: Alien: Isolation is a first-person survival horror game where the opposing alien was designed to harass
the player without using an optimal strategy that would always kill the player directly. The enemy alien spends
the game hunting the player, displaying behaviors of seeking the player’s location (a weak version of AI is
Visualized), and gradually learning from tactics the player uses repeatedly (an oppositional application of AI as
Trainee). By having players continually reason on what the alien has learned and where it will go the player is
forced to consider the state of the AI and (after repeated play) the processes involved in the AI learning.
AI DESIGN PATTERNS

60. AI TO CREATE
GAMES!!!!

63. AI TO DESIGN GAMES /
COMPUTATIONAL
CREATIVITY

64. RESOURCES
▸ IEEE Computational Intelligence and Games (CIG)
▸ AAAI Artificial Intelligence in Interactive Digital
Entertainment (AIIDE)
▸ Foundations of Digital Games (FDG)
▸ IEEE Transactions on Games (ToG)
▸ Yannakakis and Togelius: Artificial Intelligence and Games
www.gameaibook.org

65. THANK YOU FOR YOUR
ATTENTION.
JOHANNA PIRKER, JPIRKER@MIT.EDU, @JOEYPRINK
Further information:
jpirker.com
This is how others play your game!