This is the 7th of an 8 lecture series that I presented at University of Strathclyde in 2011/2012 as part of the final year AI course. This lecture covers ways that we can use AI to manage the experience that the player receives. Topics include Immersive Worlds, Player/Game Interactions, Interactive Fiction and "AI Directors" such as that found in Left4Dead

1. AI for
Experience Management

2. Context
• We talked last week about tailoring games to
specific players through content generation.
• How else can we drive engagement with the player?
• Talking today about ways the player engages with
games.
‣ How we can use AI to aid that process.
2

3. Monopoly
• Imagine you’re winning a game of Monopoly with
some friends.
• You are playing the role of a successful business
tycoon with a giant stack of cash.
• Couldn’t you put a bounty on your opponents?
• Hire arsonists to drive property value down?
3

4. Limited Interactions
• Games typically use a fixed set of rules.
• Limits how we can interact with the game world.
• Constrained by how the designer envisaged the
game be played.
• Inability to step beyond the game rules.
• In many cases the way to win is to think like the
designer.
4

5. Designer-Based Reasoning
5

6. The Greatest Game Ever
(...Maybe)
6

7. What is
Dungeons and Dragons?
• Tabletop role playing game.
• Pen and paper (and imagination) based.
• Groups take the role of adventurers in a Lord of
the Rings style setting.
• Games may take hours to play, usually based around
some sort of quest structure.
‣ E.g. Retrieve the sword of whoever from the tomb of
whoever.
7

8. The Dungeon Master
• One player takes the role of the DM
• In charge of running the game
‣ Narrator
‣ Arbitrator
‣ “AI” system for enemies
• Responsible for ensuring that
8

9. Interactions in D&D
• Dungeons and Dragons doesn’t have rules per se
• Defines a framework for interaction.
• DM can interpret the framework to suit unseen /
unplanned interactions.
‣ A player wants to climb the dungeon wall looking for
secret passages.
- This will require an athletic skill check and a dungeoneering
check
- Key is to let the players improvise and say “Yes and...”
9

10. Zork
• Old (oldoldold) school game from the 70s
• Textbased adventure game with text input
• Incorporates some language parsing systems.
‣ “Hit troll” vs “Hit the troll with the Elvish sword”
‣ Parses a variety of verbs, nouns and conjunctions
• Good HCI
• Still a very constrained human/game interaction
10

11. Starship Titanic
• Game from 1998 designed by Douglas Adams
• Starship Titanic crashes into player’s house, player
boards to try to figure out cause of disaster
• Mystery game in the style of Monkey Island
• Complete freedom to interact with characters using
regular English. Robust parsing and response system
• Tight constraints on lateral thinking
11

12. Scribblenauts
• 2009 for Nintendo DS (Scribblenauts Remix - 2011)
• Casual puzzle game based around collecting stars
• In order to collect the stars, solve puzzles.
• Solution generally involves “summoning” an object.
• Objects have distinct characteristics.
12

13. Scribblenauts
13

14. Scribblenauts
14

15. Scribblenauts
15

16. Scribblenauts
16

17. Scribblenauts
17

18. Scribblenauts
18

19. Scribblenauts
• 22,802* objects in the Scribblenauts database
• Object characteristics
‣ Art
‣ Health
‣ AI Behaviour
‣ Animations
‣ Interactions with other objects
• Adjectives give scope for lots of customisation
19

20. Scribblenauts
• Using the object database frees the player to tackle
problems however they want.
• In effect, the database becomes the framework for
interactions.
• Sufficient variety that the player doesn’t perceive
constraints.
• Encourages lateral thinking
20

21. Scribblenauts
21

22. Scribblenauts
22

23. Scribblenauts
23

24. Scribblenauts
24

25. Scribblenauts
25

26. Scribblenauts
26

27. Scribblenauts
27

28. Freeform Interaction
• Allows the player to solve problems in their own
way.
• Massive sense of satisfaction.
• Moves away from “thinking like the designer”
• Works in Scribblenauts primarily because of
simplistic environment
28

29. Immersive Worlds
• We can engage with Scribblenauts well because,
although simple, the depth of the world makes it
immersive.
• Limits interactions in other ways
‣ No unfolding story
‣ No characters to interact with
• What if we looked at breadth of engagement rather
than depth?
29

30. Skyrim
• Mentioned Skyrim last week.
• Amazing breadth to the world.
‣ Talked about the option to join Thieves Guild group.
- Also Companions, Dark Brotherhood, Bard’s guild, College of
Magic.
‣ 18 Skill disciplines, each with ~10 individual “perks”
‣ About 300 locations within the world to explore
• Massive “living” world to play in
30

31. Skyrim - Economy
• Cities in Skyrim maintain a (simplistic) economic
model.
• Affects availability of merchandise, amount of money
NPCs have and prices of goods.
• Player can affect the economy by building up (or
tearing down) local business.
31

32. Detour
• “Raid on Bungeling Bay” (1984) involved a
helicopter attacking factories in a city.
• City infrastructure was intricately simulated.
• Disrupting supply lines could cripple infrastructure
and in turn, limit military forces counter-attacking
• Level editor for this game would later be released
as “Sim City”.
32

33. Skyrim - Story
• Skyrim has multiple plots that run in parallel.
• Option to progress whichever you feel like.
• Each group has a storyline as well as Radiant-
generated side quests.
• Different plots can influence each other
‣ One character appears in two plots, one as a core character,
another as a guest at a party
‣ Extra interaction options at the party if you already know her
33

34. Skyrim - Consequences
• Quests and conversations have impact.
• Positive and negative responses and attitudes.
‣ Characters build up a model of the player
• Callbacks - the outcome of a quest will be
referenced at a later date.
34

35. Skyrim - Conclusions
• The HGI here is not sophisticated.
• Relies heavily on designer-led thinking.
‣ Can only interact with what has been anticipated
• Few solutions to problems based on lateral thinking.
‣ Example - Can’t make a pile of brooms into a ladder
• Implicitly a shallow experience.
• Compensates by providing a very broad diverse
world with consequences - playground of shallow.
35

36. Immersive Worlds
• Immersive worlds engage the player in a way games
like Scribblenauts can’t.
• We can virtually “inhabit” worlds.
• Emotional attachment to characters, locations and
events.
• The plot’s narrative becomes secondary to the
player-centric narrative of the world.
36

37. Interactive Fiction
• Iteractive Storytelling / Dynamic Narrative
• The story adapts to the responses from the player
• Player’s actions can shape the direction of the story
• Where AI meets Storytelling
37

38. Choose Your Own
Adventure
• Basic version of this is the Choose Your Own
Adventure books
• Page or so of text to read, then a few choices of
next action.
• Associated page number, reader skips to that page
• Rinse and repeat through a plot til an end is
reached.
38

39. Story Beats
• A “Story Beat” is a literary device.
• A small piece within a scene with an exchange
between the characters and an action / revelation
that advances the story.
• Story > Act > Scene > Beat
• Smallest unit of story
39

40. AI for Interactive Fiction
• What we’re talking about with IF is :
• Automating the Choose Your Own Adventure
process.
• Making it more subtle than “Turn to Page 14 to fall
down the giant pit”.
• Creating beats on the fly.
40

41. Branching Narrative
• Branching narrative is most similar to choose your
own adventure.
• Points in the story require the player’s input
• Determines the direction the story will take.
• Effectively a sequence of if/else
41

42. Branching Narrative
• Combinatorial explosion is a big factor here.
• If every choice has just 2 options, at the kth decision
‣ 2k possible states
• We’ve seen this before in game trees, state spaces
and planning.
42

43. Branching Narrative
• We can use procedural content systems to lessen
the amount of work to be done.
• Do we need 3 different versions of one beat which
alter just one line dependent on minor actions?
• Can we dynamically tweak a beat at runtime?
• Effectively, this is procedural content customised
based on the player’s history ingame.
43

44. Multi-Linear Narrative
• Typically media has a single linear narrative.
• There may be multiple arcs at play, but a single story
is being told.
• Games allow us to interact.
• Multiple plot lines can be set up
‣ We can explore at will
• Progress (or lack of) in one can affect another.
• Each one may itself be branching
44

45. Interactive Fiction as
Planning
• Maria introduced Planning in the first half of the
course.
• Recap
‣ Planning is the process of finding a sequence of actions
that will transform a world from a given initial state into a
state in which certain goals are true.
• What if we don’t talk about actions?
45

46. Planning with Beats
• Let’s assume we have a pre-made set of Story Beats
• We can see a narrative as a sequence of beats
stitched together.
• A traditional linear narrative is a standard planning
problem.
‣ Characters X and Y start in this situation
‣ We want to find a sequence of beats that puts them into
this final situation.
46

47. Planning with Beats
• We want to get the player to a specific point in the
story.
• We want the player to be free to choose their own
path.
• We always want to be guiding the player back
toward the point we’re trying to get to.
• This is not so straightforward.
47

48. Plan Repair
• Many assumptions related to Automated Planning
‣ When we execute a plan, it probably won’t work
• We have a whole set of research ready to go on
“what to do when things don’t work out”
• Effectively this is the same situation as when the
player doesn’t follow the path we are looking for.
48

49. Drop-In Drop-Out
Narrative
• A big goal for narrative is the ability to drop-in and
drop-out and still have the story make sense.
• Consider games that offer co-operative story lines.
‣ Halo 3, Gears of War
• Typically require a constant companion that is
controlled by the computer in single player.
• Another person can assume control.
49

50. Drop-In Drop-Out
Narrative
• Consider an online game such as WoW.
• Large raids involve up to 40.
• What happens when a sub-group is late?
• What happens when a group has to leave early?
50

51. Drop-In Drop-Out
Narrative
• Technique that allows for customisation of the plot.
• In many ways similar to Planning.
• Finding explanations for why extra players arrive.
‣ Alternate routes into a level.
• Explaining why players leave within the context of
the narrative.
‣ E.g. Banished, teleported etc.
51

52. Interactive Fiction
• IF allows us to generate worlds that feel more
engaging to players.
• The world can react to the player’s actions.
• The AI can direct the player’s attention to ensure
target “gates” are passed through in the narrative.
• Makes the world “come alive”
52

53. AI for Scenario Control
• We’ve looked at ways we can adapt the plot of the
world to match what the player is doing.
• Can we adapt the pace of the world too?
‣ Alter the flow of enemies.
53

54. Left 4 Dead
• Survival shooter game from Valve (2008)
• Everyone turned into zombies except four survivors
• Cooperative game for 1-4 players
‣ Take the role of the survivors
‣ Escape the city and find safety
• If playing short-handed, AI players will fill the team
54

55. Horror Movie Pacing
• The core thing Valve were aiming for was replicate
the dramatic pacing of a horror movie.
• This is a standard device for building suspense.
• “Emotional Intensity”
‣ Starts low, gradually builds, peaks at a high level.
‣ Tapers off to a relaxation level.
‣ Cycle repeats.
55

56. Measuring Intensity
• In order to model intensity need a range of metrics.
• Problem of feature selection we discussed earlier in
section on Dimensionality Reduction.
• Selected features for this include :
‣ Damage being dealt
‣ Damage being taken
‣ Distance to nearest zombie
• Decays over time.
56

57. Delta(Intensity)
• The desired emotional intensity matches the
current progression through the dramatic cycle.
• The current actual intensity will be different.
• This gives rise to a difference between them.
• The AI Director modulates the game world to
compensate for this difference.
57

58. The AI Director in Action
58

59. Altering the Game
• In Left 4 Dead, the principle method of altering the
game is changing the number of enemies.
‣ For peak intensity, hordes of enemies come from all sides
- Spawned by the Director
‣ At low intensity, during relaxation periods, enemies are
removed.
• Pretty basic approach - first pass at the technology
59

60. Enhanced AI Directors
• More subtle methods can be used to manage the
player’s experience.
‣ Weather - varying intensity of storms and lightning to
build intensity.
‣ Items - during intense times, reduce the amount of ammo
available.
‣ Enemy types - keep the player on their toes
60

61. Hierarchical Concurrent
State Machines
• The HCSM is the driving force behind the AI
Director.
• 1998 technique developed for scenario management
in a driving simulator.
61

62. Hierarchical Concurrent
State Machines
• Building block of an HCSM is itself an HCSM
‣ Recursive by nature
• Analogue input wire
• Configuration parameters
• Output wire
62

63. Hierarchical Concurrent
State Machines
63

64. Hierarchical Concurrent
State Machines
• Input - Delta(Intensity)
• Configuration - current parameters of level
• Objective - Minimise Delta(Intensity)
• Child HCSMs - Spawn new enemies, alter level etc.
64

65. Further Reading
• HCSMs were first introduced in
‣ “HCSM: A Framework for Behaviour and Scenario
Control in Virtual Environments” Cremer, Kearney and
Papelis
• Discussed in some detail on AIGameDev.com
‣ http://aigamedev.com/open/reviews/hcsm-concurrent-
state-machine/ (me)
65

66. Summary
• Covered “Experience Management”
• Interaction Design for Immersive Worlds
• Interactive Fiction
• Scenario Control
‣ HCSMs
66

67. Next (Final) Lecture
• What is Game AI?
• Traditional AI vs Game AI
• Emergence
• Debugging
• Software Engineering in AI
‣ Scrum
• Your AI Toolbox
67