Scripts:
Common Sense Story
Understanding
Motivations
• So far, you have used FOL, And/Or Graphs,
production rules, semantic nets to represent
knowledge.
• Try to represent your behaviors and the reasoning
behind your behaviors this morning from the time to
got up to the time you left your place.
• These AI technologies run into their limits when we
want to represent common sense knowledge and
reasoning.
• Semantic nets that you learned last can do some
limited common sense reasoning.
• In this lecture, you will learn also scripts to strengthen
your arsenal of common sense knowledge
representation.
Objectives
1. Common sense assumptions
2. Conceptual dependency theory
3. Restaurant script
4. Story understanders
A really short story
Sue went out to lunch. She sat at a table and called a
waitress, who brought her a menu. She ordered a
sandwich.
• Why did the waitress bring a menu to Sue?
• Who was the “she” who ordered a sandwich?
• Who paid?
• It is easy for us to answer these question
because we knew many assumptions not
explicitly mentioned in the story.
• How to get a computer to do the same thing?
• How to represent the daily common sense
assumptions that we know?
Basic idea of common sense
• Text: Vincent loves Mia.
• Simple predicate: loves(vincent,mia)
• Representation: x, y
vincent(x)
mia(y)
love(x,y)
• FOL: xy(vincent(x) & mia(y) & love(x,y))
• Common sense assumptions:
x (vincent(x) man(x))
x (mia(x) woman(x))
x (man(x) woman(x))
Texts and Ambiguity
• Usually, ambiguities cause many possible
interpretations
• Example:
Butch walks into his modest kitchen.
He opens the refrigerator.
He takes out a milk and drinks it.
Texts and Ambiguity
• Usually, ambiguities cause many possible
interpretations
• Example:
Butch walks into his modest kitchen.
He opens the refrigerator.
He takes out a milk and drinks it.
Texts and Ambiguity
• Usually, ambiguities cause many possible
interpretations
• Example:
Butch walks into his modest kitchen.
He opens the refrigerator.
He takes out a milk and drinks it.
Texts and Ambiguity
• Usually, ambiguities cause many possible
interpretations
• Example:
Butch walks into his modest kitchen.
He opens the refrigerator.
He takes out a milk and drinks it.
Consistency checking
• Inconsistent text:
– Mia likes Vincent.
– She does not like him.
• Two interpretations, only one consistent:
– Mia likes Jody.
– She does not like her.
– Who does not like whom?
– Jody does not like Mia.
Endow a computer with common sense
• How do we get the computer to
– disambiguate a sentence?
– sort out inconsistencies?
– know common sense?
• One attempt is to standardize the semantic
network for the English language.
• Verb-oriented approach and concept
dependency theory are such attempts.
• They parse a sentence by focusing on the
verb.
Verb-oriented approach
• Single out the main verb (action) of the
sentence.
• This is the central node of the net.
• Links at this node are related to one of the 5
cases:
1. agent
2. object
3. instrument
4. location
5. time
Case frame representation of the sentence
“Sarah fixed the chair with glue.”
Concept dependency theory
• Arrow indicates direction of dependency
• Double arrow indicates agent-verb
relationship
• p = Past tense
• o = Object case relation
• R = Recipient case relation
“John throws the ball”
This conceptual dependency graph is stored in the computer.
It represents the canonical form for the semantic "John throws the ball".
The original sentence could have been written in English, Chinese, etc.
4 basic syntactic units
In conceptual dependency theory, there are 4 basic
syntactic units, independent of the natural
language.
1. ACT
– action, verb
2. PP, picture producer
– name, noun, pronoun
3. AA, action aider
– modifiers of actions, adverbs
4. PA, picture aider
– modifiers of objects, adjectives
Some primitive ACTs
Primitive ACTs represent all basic actions.
• ATRANS transfer a relationship give
• PTRANS transfer a physical location of an object go
• PROPEL apply physical force to an object push
• MOVE move body part by owner kick
• GRASP grab an object by an actor grasp
• INGEST ingest an object by an animal eat
• EXPEL expel from an animal’s body cry
• MTRANS transfer mental information tell
• MBUILD mentally make new information decide
• CONC conceptualize or think about an idea think
• SPEAK produce sound say
• ATTEND focus sense organ listen
“John ate the egg”
primitive
act
direction of
past dependency
tense object
relation
agent-verb
relationship
direction of
object
within action
+ This act consists
2 sub-acts.
+
Conceptual dependency graphs
PP ACT
PP PA
“John prevented Mary from giving a book to Bill”
past tense:
prevented
John causes Mary
…
conditional /
negation
direct object
past tense: indirect object
gave
recipient Bill
Summary
• Semantic networks can be used to represent
meanings.
• Conceptual dependency graphs can be used
to standardize the meaning of sentences.
• A set of these related graphs can be used to
understand simple stories (screen plays).
• Scripts technology is next. …
Answer questions about a story
John went to a restaurant, The hostess seated him. She gave him a
menu. The waiter came to the table. John ordered a lobster. He
was served quickly, left a large tip and the restaurant.
Q: What did John eat?
Lobster.
Q: Who gave John the menu?
The hostess.
Q: Who gave John the lobster?
The waiter.
Q: Who paid the check?
John.
Q: What happened when John went to the table?
The hostess gave him a menu and John sat down.
Q: Why did John get a menu?
So he could order.
Q: Why did John give the waiter a large tip?
Because he was served quickly.
Q: How much time did John spend in the restaurant, 5
minutes? half an hour? an hour? 5 hours?
Restaurant script
Sue went out to lunch. She sat at a table and called a
waitress, who brought her a menu. She ordered a
sandwich.
• Why did the waitress bring a menu to Sue?
• Who was the “she” who ordered a sandwich?
• Who paid?
• People organize background knowledge into
structures that correspond to typical situations
(scripts)
• Script: A typical scenario of what happens in…
– a restaurant
– a soccer game
– a classroom
– the morning: get up, eat breakfast, go to school
Components of scripts
1. Entry conditions, pre-conditions
– Facts that must be true to call the script
– An open restaurant, a hungry customer that has
some money
2. Results, post-conditions
– Facts that will be true after the script has
terminated
– Customer is full and has less money; restaurant
owner has more money
Components of scripts cont'
3. Props
– Typical things that support the content of the
script
– waiters, tables, menus
4. Roles
– Actions that participants perform
– Represented using conceptual dependency
– Waiter takes orders, delivers food, presents bill
5. Scenes
– A temporal aspect of the script
– Entering the restaurant, ordering, eating, …
Scene 1: Enter customer
• Script: restaurant
• Roles: customer (S), waiter, chef, cashier
• Reason: to get food so as to up in pleasure
and down in hunger
• Scene1: entering
1. S PTRANS S into restaurant
2. S ATTEND eyes to where empty tables are
3. S MBUILD mentally decides where to sit
4. S PTRANS S to table
5. S MOVE S to sit down
Scene 2: Ordering
(W brings menu)
S
Last 2 scenes
• Scene3: eating
1. Cook ATRANS Food to Waiter
2. Waiter ATRANS F to S
3. S INGEST Food
• Scene4: exiting
1. W write restaurant bill
2. W PTRANS W to S
3. W ATRANS bill to S
4. S ATRANS tip to waiter
5. S PTRANS S to cashier
6. S ATRANS money to cashier
7. S PTRANS S out of restaurant
Prolog implementation
Sue went out to lunch. She sat at a table and called a
waitress, who brought her a menu. She ordered a
sandwich.
• Invoke (call) the Restaurant script
• Check entry conditions
– Unify {S / Sue}
– Assume that (typically) Sue is hungry and
Sue has money
• Unify people and things in the story with the
roles and props in the script
– {W / waitress, F / sandwich}
Queries
• Why did the waitress bring a menu to Sue?
– Because S MTRANS “need menu” to W …
– Sue tells “need menu” to waitress
• Who was the “she” who ordered a
sandwich?
– S MTRANS “I want F” to W
– Sue tells “I want a sandwich” to the waitress
• Who paid?
– S ATRANS money to M …
– Sue gives money to the cashier
SAM
John went to a restaurant last night. He ordered steak. When he
paid he noticed he was running out of money. He hurried
home since it had started to rain.
• SAM (Script Applier Mechanism) reads in the above
story.
• Parses it into an internal conceptual dependency
representation.
• Binds the people and things in the story to roles and
props in the script.
• Use default to fill in any missing info.
• Then answer these questions:
• Did John eat dinner last night?
• How could John get a menu?
• What did John buy?
• Did John use cash or a credit card?
Successful applications
• SAM has progressed from reading simple
made-up stories to newspaper reports about
vehicle accidents, visiting dignitaries and
several other knowledge domains.
• SAM demonstrates its comprehension of a
story by summarizing or paraphrasing it, and
by answering questions about it.
• Database queries
• Chat within special domains: football, stock
market, etc.
Scripts are not so flexible
Melissa was eating dinner at her favorite restaurant
when a large piece of plaster fell from the ceiling
and landed on her date. She then heard some more
gun shots.
• Was Melissa eating a date salad?
• Was Melissa's date plastered?
• What did she do next?
• Common sense reasoning is extremely
difficult for computers.
Problems with CDGs and scripts
• Knowledge must be decomposed into fairly
low level primitives.
• Primitive acts are not necessarily what
humans do.
• Impossible or difficult to find correct set of
primitives.
• Can't produce them automatically from
natural language.
• Scripts needs to be built by hand.
• Instability: minor changes, such as
misspelling, in the system, cause drastic
downgrade in performance
• No learning systems
Conclusion
• Conceptual dependency graphs extends
semantic nets by standardizing some verbs of
the English language.
• These primitive actions are used in the
context of a scripted daily situation.
• Common sense representation and reasoning
is extremely difficult for computers.
• Some success has been achieved using
conceptual dependency theory and scripts.