Bob\'s presentation AI and Interactive Digital Entertainment (AIIDE) Conference 2010

1. U.S. Army Research, Development and Engineering Command
A perspective on the role
of AI and interactive
digital media
in military training
Presentation to the AI and Interactive Digital Entertainment Conference (AIIDE)
October 2010
Stanford University
Robert A. Sottilare, Ph.D.
Chief Technology Officer
SFC Paul Ray Smith Simulation & Training Technology Center
robert.sottilare@us.army.mil

2. Disclaimer
• I will be talking about commercial products in addition
to government-developed technology
• What I say should not be construed as an endorsement
of any commercial products
• Much of what will be covered in this talk has application
outside of a military context
• Challenge is the word of the day…
SFC Paul Ray Smith Simulation & Training Technology Center

3. Agenda
• A little about my organization and my background…
• Motivation to exploit AI and IDM for military training
• AI and IDM in serious games and military training
• Opportunities to exploit AI and IDM
• Challenges and Next Steps
SFC Paul Ray Smith Simulation & Training Technology Center

4. My Organization
SFC Paul Ray Smith
Simulation & Training Technology Center
part of the
Army Research Laboratory
SFC Paul Ray Smith Simulation & Training Technology Center

5. SFC Paul Ray Smith Simulation & Training Technology Center

6. A little about me…
• you can find me on
• 25+ years in military training & simulation
research, development and acquisition
• My research interests are in adaptive
tutoring
• my research is conducted in the Learning in
Intelligent Tutoring Environments (LITE) Lab
• and yes… I like the Simpsons
LITE Lab
SFC Paul Ray Smith Simulation & Training Technology Center

7. Motivation to exploit AI and Interactive
Media for Military Training
SFC Paul Ray Smith Simulation & Training Technology Center

8. Motivation to exploit AI and
Interactive Media for Military Training
• U.S. Army is deployed worldwide on a consistent basis
so training technology needs to be:
– accessible wherever the Soldier is located
– interactive regardless of infrastructure available
– intuitive so it can be used in the absence of instructors
• The time allotted for training is limited so training
needs to be:
– efficient as well as effective
– engaging, challenging and relevant to the Soldier’s
mission
– adaptive to the trainee’s needs and capabilities
Today, Soldiers do more than their traditional missions…
SFC Paul Ray Smith Simulation & Training Technology Center

9. Learning & Entertainment
Learning Objectives1
• Gain attention
• Inform Stimulate immersion
• Stimulate recall
Gain/maintain attention
• Present stimulus
material Motivate
• Provide learner LEARNING FUN
Guide
guidance
• Elicit performance Stimulate interaction
• Provide feedback Inform
• Assess performance
• Enhance retention
transfer
[1] Gagne, R. M. (1965). The conditions of learning. New York: Holt, Rinehart and Winston.
SFC Paul Ray Smith Simulation & Training Technology Center

10. Training
the process of bringing a person,
team or organization to an
agreed standard of proficiency
by practice and instruction
• relevant task and conditions
• standards
• measures of performance
• constructive and timely feedback
SFC Paul Ray Smith Simulation & Training Technology Center

11. Exploit AI and Interactive Digital
Media to improve Soldier…
• learning: acquisition of knowledge and/or skill
• performance: accomplishment of a given task
measured against some standard
• retention: ability to perform a learned behavior (task)
over time without the reinforcement of practice
• competence: demonstrated level of ability through
performance over time; potential for future performance
SFC Paul Ray Smith Simulation & Training Technology Center

12. Available and emerging
technologies
• intelligent technologies for training
– adaptive tutoring systems
– virtual humans
• serious games
• virtual worlds
• mobile learning
SFC Paul Ray Smith Simulation & Training Technology Center

13. Intelligent Technologies for Training
PURPOSE: Design, develop, apply and
evaluate intelligent agent technologies
(tools and methods) to enhance training,
leader development and education; and
to reduce associated support costs
OBJECTIVE: Use intelligent agents to
supplement one-to-one and one-to-many
TLE experiences where human support is Adaptive Tutors
limited, impractical or completely
unavailable
TECHNICAL CHALLENGES: Application
of intelligent agents in ill-defined domains;
understanding natural language in multi-
sided conversations; interrupted
conversations team tutoring; rapid
authoring; and virtual human cognition
Virtual Humans
SFC Paul Ray Smith Simulation & Training Technology Center

14. Intelligent Technologies for Training
use real behaviors and
physiological use simulated
measures to model and cognitive states to
predict trainee drive virtual human
cognitive states behaviors
Adaptive Tutors Virtual Humans
SFC Paul Ray Smith Simulation & Training Technology Center

15. Adaptive Tutoring Research
PURPOSE: Enable computer-based intelligent
tutors to adapt to trainee “readiness-to-learn”
by evaluating the influence of performance
variables in the trainee model
OBJECTIVE: Experimentally evaluate trainee
state variables (self-reported, observed,
physiological and behavioral) to accurately/
passively assess the trainee’s cognitive state
& adapt content/interaction to meet trainee
needs
TECHNICAL CHALLENGES: Determining
optimal sensor suite to accurately/passively
identify cognitive state; development of machine
learning models to adapt instruction
sensors  predicted trainee state  instructional strategy selection
SFC Paul Ray Smith Simulation & Training Technology Center

16. Individual Tutoring Systems
Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In
Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010.
SFC Paul Ray Smith Simulation & Training Technology Center

17. Team Tutoring Systems
Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In
Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010.
SFC Paul Ray Smith Simulation & Training Technology Center

18. Adaptive Tutoring Systems
• Few military training simulations today implement
intelligent tutoring…
• In most of those cases, the trainee model is mainly
focused on performance goals and know little or
nothing about:
– trainee cognition (including affect)
– trainee past performance (competence level)
• Many of today’s tutors focus in well-defined domains
(e.g. physics, math) and the world is messy…
• The difficult work is ahead of us…
SFC Paul Ray Smith Simulation & Training Technology Center

19. Virtual Humans
SFC Paul Ray Smith Simulation & Training Technology Center

20. Virtual Humans
• What parts of real people are represented in virtual
humans?
– physical appearance, behaviors and interaction
– limited cognition (including affect)
– culture, values?
– goals, beliefs
– social interaction
– trust
– persistence…
Arnold, Lazarus, Frijda, Scherer, Ortony et al.
SFC Paul Ray Smith Simulation & Training Technology Center

21. Appraisal Theory
courtesy of Stacy Marsella, Institute for Creative Technologies
Arnold, Lazarus, Frijda, Scherer, Ortony et al.
SFC Paul Ray Smith Simulation & Training Technology Center

22. Virtual Humans
• Many current and potential applications:
– training partners, adversaries or neutral characters
– tutoring, mentoring, coaching
– marketing
– social support
– entertainment
SFC Paul Ray Smith Simulation & Training Technology Center

23. Virtual Humans
courtesy of Jon Gratch, Institute for Creative Technologies
SFC Paul Ray Smith Simulation & Training Technology Center

24. Virtual Patients/Virtual Sick Call
• Create medically accurate virtual patients
with advanced interaction capabilities
• Train Combat Medics on the proper
procedures for sick call
TECHNICAL CHALLENGES:
• Identifying/creating appropriate and meaningful medical dialog/content
• Developing adaptive, realistic human behaviors
SFC Paul Ray Smith Simulation & Training Technology Center

25. Serious Games
SFC Paul Ray Smith Simulation & Training Technology Center

26. Tactical Combat Casualty Care
SFC Paul Ray Smith Simulation & Training Technology Center

27. Tactical Questioning System
PURPOSE:
Develop a capability for sustainment
training that included interrogations of
detainees, civilians and persons of
interest
OBJECTIVE:
• Develop 3D Interactive environment
• Soldier interacts with virtual humans
to sustain their interrogation skills in
realistic manner
• Utilize translator and foreign
languages
TECHNICAL CHALLENGES:
• Speech and natural language processing
• Question and Answer type of conversations
• Authoring capability to create new characters
SFC Paul Ray Smith Simulation & Training Technology Center

28. Multi-national Training Experiment
using Virtual Battlespace 2
• uses commercial game
• primarily kinetic operations
• distributed shared environment
SFC Paul Ray Smith Simulation & Training Technology Center

29. UrbanSim
PC-based game that allows Battalion and
Brigade Commanders to practice situational
understanding and operations in complex,
contemporary environments.
Key Learning Objectives:
– Develop and maintain situational
understanding
– Develop Commander’s intent
– Develop Commander’s critical information
requirements
– Direct operations
– Anticipate 2nd and 3rd order effects
– Command outside the chain of command
SFC Paul Ray Smith Simulation & Training Technology Center

30. Bi-lateral Negotiation
Simulation (BiLAT)
PC, game engine-based, cognitive training tool used
for developing skills in how to plan for and conduct
bi-lateral meetings and/or negotiations
in different cultural settings
Key Learning Objectives:
Negotiation Strategy: Planning and executing
tactics that help achieve mission objectives
Cultural Awareness: Applying culturally-
appropriate interpersonal skills
Trust-Building Strategy: Developing/applying
tactics to gain negotiating partners’ trust
Challenge – combine pedagogy, game play design, interactive
research technologies, and software development to address key objectives
SFC Paul Ray Smith Simulation & Training Technology Center

31. Bi-lateral Negotiation
Simulation (BiLAT)
Social and Cultural Modeling
SFC Paul Ray Smith Simulation & Training Technology Center

32. Virtual Worlds
• Next Generation VW Platforms must be:
– scalable – work across operating systems and devices
– easily deployed
– secure
– open
– interoperable
– future proof - scale dynamically
– economically sound – good business model
David Smith, Chief Innovation Officer, Lockheed Martin Global Training and Logistics
A Virtual Worlds Architecture Framework: A Shorter Path to Mastery for All DOD Personnel
SFC Paul Ray Smith Simulation & Training Technology Center

33. OnLine Interactive
Virtual Environment (OLIVE)
SFC Paul Ray Smith Simulation & Training Technology Center

34. Multinational Training Experiment
using OLIVE
SFC Paul Ray Smith Simulation & Training Technology Center

35. Enhanced Dynamic Geo-social
Environment (EDGE)
Blends traits from MMO, serious games and virtual world technology
to replicate the operational environment as accurately as possible.
SFC Paul Ray Smith Simulation & Training Technology Center
3

36. Federal Virtual Worlds Challenge
• What is the Federal Virtual Worlds Challenge?
• It is an open, global Challenge to the world for the best of
the best implementations within virtual environments.
• The event is conducted to explore innovative and interactive
solutions in virtual environments. The criteria are
intentionally unbounded to encourage creative results.
• This year the focus is on artificial intelligence!
• http://www.fvwc.army.mil/index.php
SFC Paul Ray Smith Simulation & Training Technology Center

37. Mobile Learning
SFC Paul Ray Smith Simulation & Training Technology Center

38. Combat Medic Card Games
SFC Paul Ray Smith Simulation & Training Technology Center

39. SFC Paul Ray Smith Simulation & Training Technology Center

40. Combat Hunter Mobile
SFC Paul Ray Smith Simulation & Training Technology Center

41. Challenges and Next Steps
• Economic challenges
• AI for training challenges
• IDM for training challenges
• Interaction design challenges
• Grand challenges for educational technology
• Next steps for educational technology
SFC Paul Ray Smith Simulation & Training Technology Center

42. Economic Challenges
• The U.S. Army is a large population distributed
worldwide and covering a variety of occupational
specialties
– Maintaining the proficiency of this population is challenging
and expensive
– New missions = new training content
– New personnel and equipment = continuous training
• AI technology (tools and methods) are needed to:
– reduce training costs
– while maintaining training effectiveness
– and improving availability and adaptability
SFC Paul Ray Smith Simulation & Training Technology Center

43. AI for Training Challenges
• Adaptable tutoring systems
– improve perception/prediction of
trainee’s state
– improve assessment of learning,
performance, competence and
retention
– improve selection of appropriate
instructional strategies based on the
trainee’s state (e.g. competency level)
• content, tempo, challenge, flow and
feedback
– automate authoring of content and
expert models
SFC Paul Ray Smith Simulation & Training Technology Center

44. IDM for Training Challenges
• Interactive mixed reality
training environments
– everyone wants the holodeck,
but its not economically or
technically feasible now
– explore alternative evolutionary
steps to get holodeck-like
capabilities now
– natural locomotion
– no adverse effects (e.g.
simulator sickness)
– real-time, distributed effects
SFC Paul Ray Smith Simulation & Training Technology Center

45. Interaction Design
Challenges for AI and IDM
• Trainee situational awareness
comparable to “live”
environments
• Passive machine perception
of trainee and “live”
environment
• Natural Language
Understanding & Generation
• Natural locomotion in virtual
environments
SFC Paul Ray Smith Simulation & Training Technology Center

46. Grand Challenges for
Educational Technology
• Personalize Education
• Assess Student Learning
• Support Social Learning
• Diminish Boundaries
• Develop Alternative Teaching Methods
• Enhance the Role of Stakeholders
• Address Policy Changes
Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190
SFC Paul Ray Smith Simulation & Training Technology Center

47. Next Steps for
Educational Technology
• User Modeling
• Mobile Training
– Sensing trainee state during individual training
– Interactive team training and performance assessment
• Networking Tools
• Serious Games
• Intelligent Environments
• Educational Data Mining (authoring tools)
– Content
– Models (student models, expert models…)
• Rich Interfaces
Adapted from: Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190
SFC Paul Ray Smith Simulation & Training Technology Center

48. Questions
SFC Paul Ray Smith Simulation & Training Technology Center