Understanding narrative text is more than simple information extraction on a sentence-by-sentence basis. To comprehend the true meaning of a narrative requires determining the connections between the sentences and the effect of one event on other events. This story understanding process can be greatly enhanced by the use of event descriptor templates that begin with the basic journalistic questions of who, what, when, where, why, and how but that go beyond these simple basics to address more complex relationships: role playing, context, impact, causality, and interests. Previously, representing story narratives as knowledge representations has required intensive manual effort on the part of trained knowledge engineers to correctly encode the contents of stories into a knowledge base (KB). For large volumes of text, this becomes impractical, limiting the usefulness of KB-based systems in question-answering. This paper describes a means of automating the narrative representation process by using event descriptor templates to elicit critical narrative information to be encoded in a knowledge based system.

1. Event Templates for Improved Narrative Understanding
in Question Answering Systems
Maureen Caudill
SAIC
Enterprise Information Systems Division
10260 Campus Point Drive, MS C-2
San Diego, CA 92121
Barbara Starr
SAIC
Enterprise Information Systems Division
10260 Campus Point Drive, MS C-2
San Diego, CA 92121
ABSTRACT simple look-up questions. If the system is capable of
answering complex queries that require reasoning about
Understanding narrative text is more than simple the situation, its comprehension of events in the real
information extraction on a sentence-by-sentence basis. world must substantially exceed that of a simple
To comprehend the true meaning of a narrative requires information extraction system.
determining the connections between the sentences and
the effect of one event on other events. This story Information extraction systems typically deal with
understanding process can be greatly enhanced by the narrative texts on a sentence-by-sentence basis. That is,
use of event descriptor templates that begin with the they rarely are capable of understanding the relationship
basic journalistic questions of who, what, when, where, between sentences. This limited knowledge makes them
why, and how but that go beyond these simple basics to into systems that do little more than key word matches
address more complex relationships: role playing, when trying to answer questions. In a query about
context, impact, causality, and interests. Previously, houses, a typical information extraction system looks for
representing story narratives as knowledge the word “house” in sentences. A slightly more
representations has required intensive manual effort on sophisticated version might also include synonym
the part of trained knowledge engineers to correctly searches, and include searches for “apartment,”
encode the contents of stories into a knowledge base “condominium,” “condo,” “dwelling,” “abode,” etc.
(KB). For large volumes of text, this becomes However, none of these techniques can identify
impractical, limiting the usefulness of KB-based systems sentences that refer to houses indirectly such as the one
in question-answering. This paper describes a means of below:
automating the narrative representation process by using
event descriptor templates to elicit critical narrative I lived on Main Street when I was a child, in a huge old
information to be encoded in a knowledge based system. barn. It had twelve rooms, five fireplaces, and many odd
nooks and corners where a child could disappear for
Keywords: Automating Knowledge Generation, , hours at a time.
Knowledge Engineering Bottleneck, Knowledge-Based
System Development, Question-Answering Systems, In this example, the word “barn” likely would not be
Event extraction, Event understanding, Automatic Event identified as a metaphor for “house.” Even if that
Extraction. relationship were identified, the second sentence,
1. BACKGROUND referring to the dwelling place only as “It,” would
certainly be missed by all but a very few current systems.
A question-understanding system that can answer Thus, if the question is “How many rooms were in the
questions about historical or current events must have an house where you lived as a child?” an information
understanding of those events to do more than respond to extraction system could not answer it, even though the

2. information requested is explicitly included in the The concept of prefabricated understanding of specific
sentence. For information that is not explicit, but types of events can, however, be extended to more
available only through inferencing, typical information general event types using an event descriptor template.
extraction systems have no hope of generating an The initial version of this template first seeks to answer
answer. the basic journalistic queries of who, what, when, where,
how, and why.
In knowledge-based question-answering systems,
however, the details of the house could be elicited during When viewing the output from a basic information
the knowledge representation process. In previous extraction system, some of these journalistic queries
development of knowledge-based systems, the process of appear to be quite simple to determine. For example, the
producing these knowledge representations from the question of what happened is frequently identified by the
content of a narrative has been entirely manual. Trained verb in the sentence (or in the clause). However, a single
knowledge engineers and/or experts in a particular sentence can encode more than one event, and multiple
domain have been required to manually write the sentences can encode the same event. Thus, any
knowledge representations that reflect the contents of automated system to perform story understanding must
documents. This is sometimes called the “knowledge be able to view all sentences in the story and determine
engineering bottleneck.” which portions of each refer to which specific events.
In the current paper, we describe a method of resolving For example, consider the sentence:
both these problems. The use of event descriptor [Manila, Philippines] About 36 US Special Forces
templates as models for representing narrative text troops started a month of anti-terrorism training with
provides both completeness of coverage and a method of their Filipino counterparts at a northern army base
automating the knowledge extraction and representation Tuesday, a Philippine army officer said. (taken from an
processes. article on MSNBC.com)
The following sections describe, first, the types of The obvious verb here is “started,” as in, the troops
templates that we have found useful in describing started anti-terrorism training. However, the actual main
narratives, and then a system design that automatically verb of the sentence is “said.” The verb “started” is
fills in these event templates from large volumes of text merely describing what was “said.”
documents without requiring knowledge engineers or
domain experts to generate the knowledge Sentences such as this are very common in real-world
representations. story understanding. In such cases, both verbs identify
events, one of which is an act of starting something (i.e.,
2. BASIC JOURNALISTIC RELATIONS “starting to train”) and the other is an act of
communicating (i.e., “said”). This sentence thus encodes
Much of the information needed to correctly respond to two separate events, the starting event and the saying
user queries is found in the contents of story narratives. event. This sentence can thus begin to be encoded by
These text items range from historical event summaries generating the simplest knowledge relations:
to news stories. Consequently, narrative text constitutes
an important set of data items to be able to automatically (defobject Event-E1
process and convert into knowledge representations. (instance-of Event-E1 saying))
Story narratives can be encoded in many ways, going (defobject Event-E2
back to the concept of scripts, in which a standardized set (instance-of Event-E2 starting))
of roles and activities occur in a known general
environment. The classic example of scripts is eating at Thus, the verb(s) defines the type of action that
a restaurant. There, a person who understands the basic corresponds to that event.
“restaurant script” knows how to interpret such activities
as being shown to a table and handed a menu, ordering The next simple journalistic question is who performed
from a waiter or waitress, having the meal delivered, the event. Typically, this is identified as the subject of
receiving a check, and paying for the meal. By encoding the verb for each verb in the sentence. (Care must, of
this and other stereotypical scenarios in a standard course, be taken for passive voice and similar non-typical
knowledge system, then comparing a textual narrative of sentence constructs.) Thus, a parse of the sample
a specific “eating at a restaurant” incident to the sentence above can identify the agents involved by
restaurant script, a knowledge-based system can respond identifying the subjects of each of the two verbs. This
to many complex, non-obvious questions about that enhances the definitions of the two events by adding
event. more relations:

3. (defobject Event-E1
(instance-of Event-E1 saying) (defobject Event-E1
(performed-by Event-E1 Philippine-army-officer)) (instance-of Event-E1 saying)
(defobject Event-E2 (performed-by Event-E1 Philippine-army-officer)
(instance-of Event-E2 starting) (location-of Event-E1 city-of-manila))
(performed-by Event-E2 us-special-forces-troops)
(performed-by Event-E2 filippino-troops)) Should this location not be in the base ontology, it would
of course also have to be defined.
To get these encodings, however, additional definitions
are required to identify the specific agents, since it is The next journalistic question is when did the event take
unlikely that any general ontology would have the place. This is again easier to determine for the starting
concepts of Philippine-army-officer or us-special-forces- event than the saying event:
troops already present.
(defobject Event-E2
Thus, additional prior definitions are added to the (instance-of Event-E2 starting)
knowledge base: (performed-by Event-E2 us-special-forces-troops)
(performed-by Event-E2 filippino-troops)
(defobject Philippine-army-officer (location-of Event-E2 northern-army-base)
(instance-of Philippine-army-officer soldier) (took-place-when Event-E2 Tuesday))
(citizen-of Philippine-army-officer Philippines)) Presumably, a newswire article would also have a date
attached to it, so “Tuesday” could be more specifically
(defobject us-special-forces-troop defined as the most recent Tuesday prior to the date of
(instance-of us-special-forces-troop soldier) the article.
(citizen-of us-special-forces-troop united-states))
The question of when the saying event took place is
again an inference, not something that is clearly defined;
(defobject us-special-forces-troops however, it can be presumed to be not later than the date
(group-of us-special-forces-troops us-special-forces- of the newswire article itself:
troop)
(count-of us-special-forces-troops 36)) (defobject Event-E1
(instance-of Event-E1 saying)
The next journalistic query is where does this event take (performed-by Event-E1 Philippine-army-officer)
place. This becomes somewhat more difficult to (location-of Event-E1 city-of-manila)
ascertain in many cases. In the case of the starting event, (took-place-when Event-E1 before(date-of-article)))
the location is straightforward, and can be added to the
event definition in a simple manner: The two remaining journalistic questions, how and why
are not answered by the current sentence; they remain,
(defobject Event-E2 for the moment, empty relations. And, in fact, the
(instance-of Event-E2 starting) question of why an event occurs is nearly always a more
(performed-by Event-E2 us-special-forces-troops) complex relation to implement.
(performed-by Event-E2 filippino-troops)
(location-of Event-E2 northern-army-base)) 3. COMPLEX RELATIONS
Obviously, this too spawns a definition of the term A careful reading of the original sentence shows that
northern-army-base: while the descriptors developed above are accurate, they
are sorely incomplete. For example, the “starting” event
(defobject northern-army-base does not explain what exactly is being started.
(instance-of northern-army-base army-outpost) Furthermore, there’s no information regarding why this
(location-of northern-army-base (location-function event is happening, what impacts it has on other events,
(direction-fn north) Philippines))) or any of a number of other issues that a human reader
can be expected to infer easily from the text. Thus, the
The location of the saying event is far more obscure, journalistic event relations are only the starting point for
however. In this case, with a newswire article that a far more complex set of relationships.
(presumably) has a dateline attached, the assumption
would be made that the location of the saying event is the In the case of the example sentence, the first issue is to
location specified in the dateline. Thus, the location for identify what exactly is being started. The object of the
that event would be: sentence (either direct object or indirect object,

4. depending on the verb) frequently gives insight into what • Is-innocent-bystander
is being used to perform an action or what is being • Is-financial-backer
referred to in an event description. The object of the • Is-betrayer, etc.
starting clause is the phrase a month of anti-terrorism Causality Relations
training. Thus, there must be a way to encode the • Event-causes-event
concept of what is being started. • Interest-causes-event
• Contributing-factor, etc.
(defobject Event-E2 Impact Relations
(instance-of Event-E2 starting) • Event-opposes-agent <a specific agent>
(performed-by Event-E2 us-special-forces-troops) • Event-supports-agent <a specific agent>
(performed-by Event-E2 filippino-troops) • Event-opposes-interest <a specific interest>
(location-of Event-E2 northern-army-base) • Event-supports-interest <a specific interest>
(took-place-when Event-E2 Tuesday) • Supports-interests-of <an agent and its
(action-performed-on Event-E2 anti-terrorism- interest>
training)) • Opposes-interests-of <an agent and its
interest>
This in turn spawns the definition: • Action-performed-on <an object or an agent>
• Action-performed-with <an object>, etc.
(defobject anti-terrorism-training
(instance-of anti-terrorism-training training) The relations that form the event descriptor thus provide
(opposes anti-terrorism-training terrorist-group) a rich encoding of the content of a narrative text
(duration anti-terrorism-training (days 30))) description. The benefit of this approach to knowledge
representation is two-fold. First, it ensures that each
Note that the training definition includes the approximate event is scrutinized to derive the maximum possible
duration of the training. Notice also that because the information from the narrative text. Second, it allows
prefix “anti” means “against,” the relation “opposes” is multi-sentence descriptions. For example, suppose the
used. If there were no such prefix, and the phrase was next sentence in the text is:
“terrorism training,” the relation “supports” would be
used, on the assumption that training generally supports The training will provide experience in fighting bio-
the activity it concerns. terrorist attacks in a jungle environment.
The saying event also must be more carefully defined. This sentence can easily be determined to refer to the
What exactly did the Philippine army officer say? same event as the “starting” event. The subject of the
Basically, the officer described the training event. Thus, sentence is “the training.” Since it immediately follows
a reasonable encoding of the object of the saying event a sentence that refers to starting a training event, it can
is: be inferred that the training referred to is one and the
same training. Thus, it is a simple matter to amend the
(defobject Event-E1 earlier description of the training to assert:
(instance-of Event-E1 saying)
(performed-by Event-E1 Philippine-army-officer) (defobject anti-terrorism-training
(location-of Event-E1 city-of-manila) (instance-of anti-terrorism-training training)
(took-place-when Event-E1 before(date-of-article)) (opposes anti-terrorism-training terrorist-group)
(action-performed-on Event-E1 Event-E2)) (duration anti-terrorism-training (days 30))
(location-of anti-terrorism-training jungle)
This single sentence does not yield other relations. But (opposes anti-terrorism-training bio-terrorist-attack))
there are many other relations that should be considered
before abandoning the analysis of the sentence (and the Filling in such detailed event descriptors amounts to pre-
article). These relations can be grouped into categories answering a number of queries about the narrative. Any
as shown below with a few examples for each category. question that asks about these events, even in elliptical
terms, will quickly generate the correct answer. This
Role Relations removes a great deal of the processing and inferencing
• Is-aggressor burden from the question-answering system and enables
• Is-defender it to provide answers accurately, quickly, and efficiently.
• Is-protector 4. AUTOMATING THE PROCESS
• Is-mediator
• Is-peacekeeper The remaining question is how can constructing such
• Is-victim event descriptors be automated. In essence, this requires

5. constructing a pre-answering system that infers the and lexical terms and to define new ones as needed.
answer to obvious questions about a narrative and stores These new terms comprise a mini-ontology that is
those answers in the knowledge base for rapid retrieval. relevant primarily to the current document; however, as
appropriate, new terms can be added to the global
The approach we have taken to this is a modular one for lexicon, thus enabling cross-document processing of
the sake of easy construction, extension, and events. Thus, a sequence of events described in multiple
maintenance. In effect, each relation or logical set of documents is an easy extension of this concept.
relations has its own miniature knowledge-based
reasoner—a knowledge base partition—which knows Furthermore, since each relation can be documented with
only how to inspect a parsed sentence and construct a the specific text that generated that relation, it is
specific relation from that sentence—if and only if that similarly easy to identify which document events and
sentence contains the necessary information to construct mini-ontologies correspond to a particular term. For
the relation. example, if a query asks about US Special Forces Troops
in the Philippines, it is very easy to determine exactly
These knowledge-based reasoners thus are kept within a which document(s) have information about these troops,
small scale, they can be called or not as appropriate for a since they are mentioned in those documents that include
given sentence, and they construct an appropriate that ontological term. A separate database (not shown in
relation (or small number of similar relations) based on the figure) keeps track of which ontological terms are
the contents of that sentence. Figure 1 illustrates how the defined in which document files and mini-ontologies.
architecture operates in the case of the example sentence. Thus, if a query arrives asking about a specific concept
(such as the U.S. Special Forces Troops in the
The KB Controller is the coordinator for the event Philippines), only those knowledge bases that include
descriptor construction. It does not generate event that concept are added to the answer reasoner’s
relations itself, but calls whatever other modules are knowledge base to serve as the basis for determining the
appropriate in a given sentence to determine the answer. This provides a mechanism for automatic
relations. It also deals with cross-sentence descriptors by partitioning of very large scale knowledge bases as well
determining that the current sentence probably refers to as increases the efficiency of the answering system.
an event that is already described; in this case it tells the
various KB partitions it calls not to define a new event The modular architecture enables each KB partition to
but rather to modify a specific existing event. focus on only a single problem rather than trying to
generate the entire event descriptor. This makes the
Each of the KB partitions has access to the global entire system easy to construct, easy to extend, and easy
ontology and lexicon to identify appropriate concepts to maintain.
Parsed sentence KB
Controller KB Ontology
Calls KB & Lexicon
Engines
as needed
Etc.
Instance-of Performed-by Location-of When Roles Causality
KB partition KB partition KB partition KB partition KB partition KB partition
(defobject Event-E2 (location-of Event-E2
(instance-of Event-E2 northern-army-base))
starting)) including definition of terms
(defobject Event-E2 (took-place-when Event-
(performed-by Event-E2 E2 Tuesday)
us-special-forces-troops))
including definitions for Figure 1. Architecture for the Event Descriptor system.
“us-special-forces-troops”

6. 5. CONCLUSIONS and Computational Issues (ICCS 2000). Darmstadt,
Germany. August 14-18, 2000.
This paper has outlined a workable methodology for
constructing a narrative understanding tool for text [10] McIlraith, S. and Amir, E. “Theorem Proving with
narratives that can automatically comprehend stories and Structured Theories,” Proceedings of the
answer questions about those stories. The system is Seventeenth International Conference on Artificial
under construction and preliminary results indicate that it Intelligence (IJCAI-01). pp. 624 -- 631, August, 2001.
is flexible, powerful, and highly extensible.
[11] Amir, E. and McIlraith, S. “Partition-Based Logical
6. REFERENCES Reasoning for First-Order and Propositional Theories,”
Submitted for Publication.
[1] Adam Farquhar, Richard Fikes, and James P. Rice.
“A Collaborative Tool for Ontology Construction.,” This work was supported by the Advanced Research and
International Journal of Human Computer Studies, Development Activity (ARDA) as part of its AQUAINT
46:707-727, 1997. Program. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of
[2] Peter D. Karp, Vinay K. Chaudhri, and Suzanne M. the author(s) and do not necessarily reflect the views of
Paley. “A Collaborative Environment for Authoring the U.S. Government.
Large Knowledge Bases.” Journal of Intelligent
Information Systems, 1998.
[3] Paul Cohen, Robert Schrag, Eric Jones, Adam Pease,
Albert Lin, Barbara Starr, David Gunning, and Murray
Burke. The DARPA High Performance Knowledge
Bases Project. AI Magazine, Winter, 1998. pp. 25-49
[4] B. Katz, “From Sentence Processing to Information
Access on the World Wide Web,” AAAI Spring
Symposium on Natural Language Processing for the
World Wide Web, Stanford University, Stanford CA
(1997).
[5] Gentner, D. and K. Forbus “MAC/FAC: A Model of
Similarity-based Retrieval,” Proceedings of the
CognitiveScience Society. 1991.
[6] Forbus, K. and D. Oblinger “Making SME Greedy
and Pragmatic,” Proceedings of the Cognitive Science
Society. 1990.
[7] V. K. Chaudhri, J. D. Lowrance, M. E. Stickel, J. F.
Thomere, and R. J. Waldinger
“Ontology construction toolkit,” Technical Note
Ontology, AI Center, SRI International, 333
Ravenswood Ave., Menlo Park, CA 94025, 2000.
[8] Deborah McGuinness “Description Logics Emerge
from Ivory Tower”’ Stanford Knowledge Systems
Laboratory Technical Report KSL-01-08 2001. In the
Proceedings of the International Workshop on
Description Logics. Stanford, CA, August 2001.
[9] Deborah McGuinness “Conceptual Modeling for
Distributed Ontology Environments.”(Word format)
To appear in Proceedings of the Eighth International
Conference on Conceptual Structures Logical,Linguistic,