; Compiling a Lexicon of Cooking Actions for Animation Generation
Learning Center
Plans & pricing Sign in
Sign Out

Compiling a Lexicon of Cooking Actions for Animation Generation


  • pg 1
									    Compiling a Lexicon of Cooking Actions for Animation Generation
                         Kiyoaki Shirai                 Hiroshi Ookawa
                       Japan Advanced Institute of Science and Technology
                         1-1, Asahidai, Nomi, 923-1292, Ishikawa, Japan

                    Abstract                          the Animation of Cooking Tasks), which analyzed
                                                      verbal modifiers to determine several features of
   This paper describes a system which gen-           an action, such as the aspectual category of an
   erates animations for cooking actions in           event, the number of repetitions, duration, speed,
   recipes, to help people understand recipes         and so on (Karlin, 1988). Uematsu developed
   written in Japanese. The major goal of this        “Captain Cook,” which generated animations from
   research is to increase the scalability of the     cooking recipes written in Japanese (Uematsu et
   system, i.e., to develop a system which can        al., 2001). However, these previous works did
   handle various kinds of cooking actions.           not mention the scalability of the systems. There
   We designed and compiled the lexicon of            are many linguistic expressions in the cooking do-
   cooking actions required for the animation         main, but it is uncertain to what extent these sys-
   generation system. The lexicon includes            tems can convert them to animations.
   the action plan used for animation genera-            This paper also aims at developing a system to
   tion, and the information about ingredients        generate animations from cooking recipes written
   upon which the cooking action is taken.            in Japanese. We especially focused on increasing
   Preliminary evaluation shows that our lex-         the variety of recipes that could be accepted. After
   icon contains most of the cooking actions          presenting an overview of our proposed system in
   that appear in Japanese recipes. We also           Subsections 2.1 and 2.2, the more concrete goals
   discuss how to handle linguistic expres-           of this paper will be described in Subsection 2.3.
   sions in recipes, which are not included
   in the lexicon, in order to generate anima-        2 Proposed System
   tions for them.                                    2.1 Overview

1 Introduction                                        The overview of our animation generation sys-
                                                      tem is as follows. The system displays a cooking
The ability to visualize procedures or instruc-       recipe in a browser. As in a typical recipe, cooking
tions is important for understanding documents        instructions are displayed step by step, and sen-
that guide or instruct us, such as computer manuals   tences or phrases representing a cooking action in
or cooking recipes. We can understand such docu-      the recipe are highlighted. When a user does not
ments more easily by seeing corresponding figures      understand a certain cooking action, he/she can
or animations. Several researchers have studied       click the highlighted sentence/phrase. Then the
the visualization of documents (Coyne and Sproat,     system will show the corresponding animation to
2001), including the generation of animation (An-     help the user understand the cooking instruction.
dre and Rist, 1996; Towns et al., 1998). Such ani-       Note that the system does not show all proce-
mation systems help people to understand instruc-     dures in a recipe like a movie, but generates an
tions in documents. Among the various types of        animation of a single action on demand. Further-
documents, this research focuses on the visualiza-    more, we do not aim at the reproduction of recipe
tion of cooking recipes.                              sentences in detail. Especially, we will not prepare
   Many studies related to the analysis or genera-    object data for many different kinds of ingredients.
tion of cooking recipes have been done (Adachi,       For example, suppose that the system has object
1997; Webber and Eugenio, 1990; Hayashi et al.,       data for a mackerel, but not for a sardine. When
2003; Shibata et al., 2003). Especially, several      a user clicks the sentence “fillet a sardine” to see
researchers have proposed animation generation        the animation, the system will show how to fillet a
systems in the cooking domain. Karlin, for exam-      “mackerel” instead of “sardine”, with a note indi-
ple, developed SEAFACT (Semantic Analysis For         cating that the ingredient is different. We believe
      Input sentence
(ex. chop an onion finely)                                imation Generator interprets the action plan and
                             Lexicon of Cooking Actions   produces the animation.
      Action Matcher           Basic Action 1
                                 ``fry''                  2.3 Goal
                                  action plan
                                                          The major goals of this paper are summarized as
       Action Plan             Basic Action 2             follows:
                                 ``chop finely''
                                  action plan
        Animation                                         G1. Construct a large-scale lexicon of cooking ac-
        Generator                                             tions
                                                               In order to generate animations for various
       Animation                                               kinds of cooking actions, we must prepare a
                                                               lexicon containing many basic actions.
             Figure 1: System Architecture
                                                          G2. Handle a variety of linguistic expressions
that the user will be more interested in “how to fil-           Various linguistic expressions for cooking ac-
let” than in the specific ingredient to be filleted.             tions may occur in recipes. It is not realistic
In other words, the animation of the action will be            to include all possible expressions in the lex-
equally helpful as long as the ingredients are simi-           icon. Therefore, when a linguistic expression
lar. Thus we will not make a great effort to prepare           in an input sentence is not included in the lex-
animations for many kinds of ingredients. Instead,             icon, the system should calculate the similar-
we will focus on producing the various kinds of                ity between it and the basic action in the lex-
cooking actions, to support users in understanding             icon, and find an equivalent or almost similar
cooking instructions in recipes.                               action.

                                                          G3. Include information about acceptable ingre-
2.2     System Architecture
                                                              dients in the lexicon
Figure 1 illustrates the architecture of the proposed          Even though linguistic expressions are the
system. First, we prepare the lexicon of cooking               same, cooking actions may be different ac-
actions. This is the collection of cooking actions             cording to the ingredient upon which the ac-
such as “fry”, “chop finely”, etc. The lexicon has              tion is taken. For example, “cut into fine
enough knowledge to generate an animation for                  strips” may stand for several different cook-
each cooking action. Figure 2 shows an exam-                   ing actions. That is, the action of “cut
ple of an entry in the lexicon. In the figure, “ex-             cucumber into fine strips” may be differ-
pression” is a linguistic expression for the action;           ent than “cut cabbage into fine strips”, be-
“action plan” is a sequence of action primitives,              cause the shapes of cucumber and cabbage
which are the minimum action units for animation               are rather different. Therefore, each entry in
generation. Roughly speaking, the action plan in               the lexicon should include information about
Figure 2 represents a series of primitive actions,             what kinds of ingredients are acceptable for a
such as cutting and rotating an ingredient, for the            certain cooking action.
basic action “chop finely”. The system will gen-
erate an animation according to the action plan in           As mentioned earlier, the main goal of this re-
the lexicon. Other features, “ingredient examples”        search is to increase the scalability of the system,
and “ingredient requirement”, will be explained           i.e., to develop an animation generation system
later.                                                    that can handle various cooking actions. We hope
   The process of generating an animation is as           that this can be accomplished through goals G1
follows. First, as shown in Figure 1, the system          and G2.
compares an input sentence and expression of the             In the rest of this paper, Section 3 describes
entries in the lexicon of cooking actions, and finds       how to define the set of actions to be compiled
the appropriate cooking action. This is done by the       into the lexicon of cooking actions. This concerns
module “Action Matcher”. Then, the system ex-             goal G1. Section 4 explains two major features
tracts an action plan from the lexicon and passes it      in the lexicon, “action plan” and “ingredient re-
to the “Animation Generator” module. Finally An-          quirement”. The feature ingredient requirement is
              Basic Action 2
                  expression                                       (chop finely)
                  action plan                 cut(ingredient,utensil,location, 2)
                                              rotate(ingredient,location, x, 90)
                                              rotate(ingredient,location, z, 90)
                                              cut2(ingredient,utensil,location, 10)
                                              cut(ingredient,utensil,location, 20)
                   ingredient examples                 (okra),          (shiitake mushroom)
                   ingredient requirement     kind=vegetable|mushroom

                    Figure 2: Example of an Entry in the Lexicon of Cooking Actions

related to goal G3. Section 5 reports a preliminary          help people, especially novices, to under-
survey to construct the module Action Matcher in             stand cooking actions in recipes. The lexicon
Figure 1, which is related to goal G2. Finally, Sec-         of cooking actions based on the cooking text-
tion 6 concludes the paper.                                  books includes many cooking operations that
                                                             novices may not know well.
3 Defining the Set of Basic Actions
                                                          3. The definition of basic actions does not de-
In this and the following sections, we will explain          pend on the module Animation Generator.
how to construct the lexicon of cooking actions.
                                                             One of the standards for the definition of ba-
The first step in constructing the lexicon is to de-
                                                             sic actions is animations generated by the
fine the set of basic actions. As mentioned earlier
                                                             system. That is, we can define basic cook-
(goal G1 in Subsection 2.3), a large-scale lexicon
                                                             ing actions so that each cooking action cor-
is required for our system. Therefore, the set of ba-
                                                             responds to an unique animation. This ap-
sic actions should include various kinds of cook-
                                                             proach seems to be reasonable for an anima-
ing actions.
                                                             tion generation system; however, it depends
3.1   Procedure                                              on the module Animation Generator in Fig-
                                                             ure 1. Many kinds of rendering engines are
We referred to three cooking textbooks or man-
                                                             now available to generate animations. There-
uals (Atsuta, 2004; Fujino, 2003; Takashiro and
                                                             fore, Animation Generator can be imple-
Kenmizaki, 2004) in Japanese to define the set of
                                                             mented in various ways. When changing the
basic actions. These books explain the fundamen-
                                                             rendering engine used in Animation Genera-
tal cooking operations with pictures, e.g., how to
                                                             tor, the lexicon of cooking actions must also
cut, roast, or remove skins/seeds for various kinds
                                                             be changed. So we decided that it would not
of ingredients. We extracted the cooking opera-
                                                             be desirable to define the set of basic actions
tions explained in these three textbooks, and de-
                                                             according to their corresponding animations.
fined them as the basic actions for the lexicon. In
other words, we defined the basic actions accord-             In our framework, the definition of basic ac-
ing to the cooking textbooks. The reasons why we             tions in the lexicon does not depend on Ani-
used the cooking manuals as the standard for the             mation Generator. This enables us to use any
basic actions are summarized as follows:                     kind of rendering engine to produce an ani-
                                                             mation. For example, when we use a poor en-
  1. The aim of cooking manuals used here is to              gine and want to design the system so that it
     comprehensively explain basic cooking oper-             generates the same animation for two or more
     ations. Therefore, we expect that we can col-           basic actions, we just describe the same ac-
     lect an exhaustive set of basic actions in the          tion plan for these actions.
     cooking domain.
                                                           We manually excerpted 267 basic actions from
  2. Cooking manuals are for beginners. The             three cooking textbooks. Although it is just a col-
     aim of animation generation system is to           lection of basic actions, we refer it as the initial
                                                        its linguistic expression was the same as one in the
       Table 1: Examples of Basic Actions
                                                        lexicon; (b2) the verbal phrase corresponded with
 expression                ingredient examples          a basic action, but its linguistic expression differed
                 (fillet)        (mackerel)              from one in the lexicon; (b3) no corresponding ba-
            (boil)                                      sic action was found in the initial lexicon, (b4) the
       (boil)                                           extracted phrase was not a verbal phrase, caused
                                   (tomato),            by error in analysis, (b5) the verbal phrase did not
 (cut into a comb shape)               (potato)         stand for a cooking action. Note that the cases in
                                     (pumpkin)          which verbal phrases should be converted to ani-
 (cut into a comb shape)                                mations were (b1), (b2) and (b3). The numbers in
                                                        parentheses (...) indicate the ratio of each case to
 (cut into a comb shape)      (turnip)
                                                        the total number of verbal phrases, while numbers
                                                        in square brackets [...] indicate a ratio of each case
lexicon of cooking actions. Table 1 illustrates sev-    to the total number of (b1), (b2) and (b3).
eral examples of basic actions in the initial lexi-        We expected that the verbal phrases in (b1) and
con. In the cooking manuals, every cooking op-          (b2) could be handled by our animation generation
eration is illustrated with pictures. “Ingredient ex-   system because the initial lexicon contained the
amples” indicates ingredients in pictures used to       corresponding basic actions. On the other hand,
explain cooking actions.                                our system cannot generate animations for verbal
                                                        phrases in (b3), which was 42.3% of the verbal
3.2    Preliminary Evaluation                           phrases our system should handle. Thus the appli-
A preliminary experiment was conducted to eval-         cability of the initial lexicon was poor.
uate the scalability of our initial lexicon of ba-
sic actions. The aim of this experiment was to          3.3 Adding Basic Actions from Recipe
check how many cooking actions appearing in real            Corpus
recipes are included in the initial lexicon.            We have examined what kinds of verbal phrases
   First, we collected 200 recipes which are avail-     were in (b3). We found that there were many gen-
able on web pages 1 . We refer to this recipe corpus    eral verbs, such as “          (add)”, “          (put
as Ra hereafter. Next, we analyzed the sentences        in)”, “          (heat)”, “         (attach)”, “
in Ra and automatically extracted verbal phrases            (put on)”, etc. Such general actions were not
representing cooking actions. We used JUMAN 2           included in the initial lexicon, because we con-
for word segmentation and part-of-speech tagging,       structed it by extracting basic actions from cook-
and KNP 3 for syntactic analysis. Finally, we           ing textbooks, and such general actions are not ex-
manually checked whether each extracted verbal          plained in these books.
phrase could be matched to one of the basic ac-            In order to increase the scalability of the lexicon
tions in the initial lexicon.                           of cooking actions, we selected verbs satisfying
   Table 2 (A) shows the result of our survey. The      the following conditions: (1) no corresponding ba-
number of basic actions was 267 (a). Among these        sic action was found in the lexicon for a verb; (2)
actions, 145 (54.3%) actions occurred in Ra (a1).       a verb occurred more than 10 times in Ra . In all,
About half of the actions in the initial lexicon did    31 verbs were found and added to the lexicon as
not occur in the recipe corpus. We guessed that         new basic actions. It is undesirable to define basic
this was because the size of the recipe corpus was      actions in this way, because the lexicon may then
not very large.                                         depend on a particular recipe corpus. However, we
   The number of verbal phrases in Ra was 3977          believe that the new basic actions are very general,
(b). We classified them into the following five           and can be regarded as almost independent of with
cases: (b1) the verbal phrase corresponded with         the corpus from which they were extracted.
one of the basic actions in the initial lexicon, and
                                                           In order to evaluate the new lexicon, we pre-
    http://www.bob-an.com/                              pared another 50 cooking recipes (Rb hereafter).
    http://www.kc.t.u-tokyo.ac.jp/                      Then we classified the verbal phrases in Rb in
    http://www.kc.t.u-tokyo.ac.jp/                      the same way as in Subsection 3.2. The results
nl-resource/knp.html                                    are shown in Table 2 (B). Notice that the ratio
                                Table 2: Result of Preliminary Evaluation
                        (A) Survey on Ra                                           (B) Survey on Rb
 (a)     # of basic actions                267                             (a)      298
  (a1)   basic actions occurred in Ra      145 (54.3%)                      (a1)    106 (35.6%)
 (b)     # of verbal phrases              3977                             (b)      959
  (b1)   basic action(same)                974   (24.5%) [28.0%]            (b1)    521   (54.3%) [62.2%]
  (b2)   basic action(dif.)               1031   (25.9%) [29.7%]            (b2)    262   (27.3%) [31.3%]
  (b3)   not basic action                 1469   (36.9%) [42.3%]            (b3)     55    ( 5.7%) [6.6%]
  (b4)   analysis error                    180    ( 4.5%)                   (b4)     45    ( 4.7%)
  (b5)   not cooking action                323    ( 8.1%)                   (b5)     76    ( 7.9%)

of the number of verbal phrases contained in the
lexicon to the total number of target verb phrases
was 94.5% ((b1)62.2% + (b2)31.3%). This is
much greater than the ratio in Table 2 (A) (57.7%).
Therefore, although the size of test corpus is small,
we hope that the scalability of our lexicon is large
enough to generate animations for most of the ver-
bal phrases in cooking recipes.

4 Compilation of the Lexicon of Basic
                                                           Figure 3: Snapshot of Generated Animation
After defining the set of basic actions for the lexi-    used VRML for animation generation. Figure 3
con, the information of each basic action must be       is a snapshot of the animation for the basic ac-
described. As shown in Figure 2, the main fea-          tion “                     (chop finely)” generated
tures in our lexicon are expression, action plan,       by our system.
ingredient examples and ingredient requirement.            Our current focus has been on the design and
The term expression stands for linguistic expres-       development of the lexicon of cooking actions,
sions of basic actions, while ingredient examples       rather than on animation generation. Implementa-
stands for examples of ingredients described in the     tion of the complete Animation Generator as well
cooking manuals we referred to when defining the         as a description of the action plans for all basic
set of basic actions. As shown in Table 1, these        actions in the lexicon are important future works.
two features have already been included in the ini-
tial lexicon created by the procedure in Section 3.     4.2 Ingredient Requirement
This section describes the compilation of the rest      Several basic actions have the same expression in
of the features: action plan in Subsection 4.1 and      our lexicon. For instance, in Figure 1, there are
ingredient requirement in Subsection 4.2.               three basic actions represented by the same lin-
                                                        guistic expression “                       (cut into
4.1   Action Plan                                       a comb shape)”. These three actions stand for dif-
For each basic action in the lexicon, the action        ferent cooking actions. The first one stands for the
plan to generate the corresponding animation is         action used to cut something like a “tomato” or
described. Action plan is the sequence of action        “potato” into a comb shape. The second stands for
primitives as shown in Figure 2. Of the 298 basic       the following sequence of actions: first cut some-
actions in the lexicon, we have currently described     thing in half, remove its core or seeds, and cut it
action plans for only 80 actions. Most of them are      into a comb shape. This action is taken on pump-
actions to cut something.                               kin, for instance. The third action represents the
   We have also started to develop Animation Gen-       cooking action for “turnip”: remove the leaves of
erator (see Figure 1), which is the module that in-     the turnip and cut it into a comb shape. In other
terprets action plans and generates animations. We      words, there are different ways to cut different in-
gredients into a comb shape. Differences among            • veg
these actions depend on what kinds of ingredients            This attribute specifies subtypes of veg-
are to be cut.                                               etables. Possible values for this attribute
   As described in Section 2.2, the module Action            are “green”, “root” and “layer”. “Green”
Matcher accepts a sentence or phrase for which a             stands for green vegetables such as
user wants to see the animation, then finds a cor-                  (spinach) and       (Chinese cabbage).
responding basic action from the lexicon. In or-             “Root” stands for root vegetables such as
der to find an appropriate basic action for a recipe                       (potato) and         (burdock).
sentence, the lexicon of cooking actions should in-          “Layer” stands for vegetables consisting of
clude information about what kinds of ingredients            layers of edible leaves such as         (let-
are acceptable for each basic action. Note that the          tuce) and           (cabbage).
judgment as to whether an ingredient is suitable
or not highly depends on its features such as kind,       • shape
shape, and components (seed, peel etc.) of the in-
                                                             This attribute specifies shapes of ingredients.
gredient. Therefore, the lexicon should include in-
                                                             The possible values are:
formation about what features of the ingredients
must be operated upon by the basic actions.
   For the above reason, ingredient requirement                 sphere, stick, cube, oval, plate, filiform
was introduced in the lexicon of cooking actions.
In this field, we manually describe the required           • peel, seed, core
features of ingredients for each basic action. Fig-          These attributes specify various components
ure 4 illustrates the three basic actions of                 of ingredients. Values are always 1. For ex-
                 (chop into a comb shape) in the             ample, “peel=1” stands for ingredients with
lexicon  4 . The basic action a1, “kind=vegetable,
shape=sphere” in ingredient requirement, means
that only a vegetable whose shape is spherical is         • instance
acceptable as an ingredient for this cooking action.         This specifies a certain ingredient, as shown
On the other hand, for the basic action a2, only a           in basic action a3 in Figure 4.
vegetable whose shape is spherical and contain-
ing seeds is acceptable. For a3, “instance=                The information about ingredient requirements
(turnip)” means that only a turnip is suitable for      was added for 186 basic actions out of the 298 ac-
this action. In our lexicon, such specific cooking       tions in the lexicon. No requirement was needed
actions are also included when the reference cook-      for the other actions, i.e., these actions accept any
books illustrate special cooking actions for certain    kind of ingredients.
ingredients. In this case, a cookbook illustrates
cutting a turnip into a comb shape in a different       4.2.2 Lexicon of Ingredients
way than for other ingredients.                            In addition to the lexicon of cooking actions, the
                                                        lexicon of ingredients is also required for our sys-
4.2.1 Feature Set of Ingredient Requirement             tem. It includes ingredients and their features such
   Here are all the attributes and possible values      as kind, shape and components. We believe that
prepared for the ingredient requirement field:           this is domain-specific knowledge for the cooking
                                                        domain. Thesauri or other general-purpose lan-
  • kind
                                                        guage resources would not provide such informa-
        This attribute specifies kinds of ingredients.   tion. Therefore, we newly compiled the lexicon
        The possible values are:                        of ingredients, which consists of only those ingre-
              vegetable, mushroom, fruit, meat,         dients appearing in the ingredients example in the
              fish, shellfish, seafood, condiment         lexicon of cooking actions. The number of ingre-
                                                        dients included in the lexicon is 93. For each entry,
        “Seafood” means seafood other than fish or       features of the ingredient are described. The fea-
        shellfish, such as    (squid),        (cod       ture set used for this lexicon is the same as that
        roe) and so on.                                 for the ingredient requirement described in 4.2.1,
       action plan is omitted in Figure 4.              except for the “instance” attribute.
               Basic Action a1
                   expression                                     (cut into a comb shape)
                   ingredient examples                (tomato),               (potato)
                   ingredient requirement      kind=vegetable, shape=sphere
               Basic Action a2
                   expression                                     (cut into a comb shape)
                   ingredient examples                  (pumpkin)
                   ingredient requirement      kind=vegetable, shape=sphere, seed=1
               Basic Action a3
                   expression                                      (cut into a comb shape)
                   ingredient examples               (turnip)
                   ingredient requirement      instance=      (turnip)

            Figure 4: Three Basic Actions of “                       (cut into a comb shape)”

   The current lexicon of ingredients is too small.    as follows:
Only 93 ingredients are included. A larger lexicon
is required to handle various recipe sentences. In       • Inconsistency in word segmentation
order to enlarge the lexicon of ingredients, we will        Word segmentation of verbal phrases in
investigate a method for the automatically acqui-           recipes, as automatically given by a morpho-
sition of new ingredients with their features from          logical analyzer, is different from one of the
a collection of recipe documents.                           basic actions in the lexicon, as shown in Fig-
                                                            ure 5 (a).
5 Matching between Actions in a Recipe
                                                            In order to succeed in matching, we need an
  and the Lexicon
                                                            operation to concatenate two or more mor-
Action Matcher in Figure 1 is the module which              phemes in a phrase or to divide a morpheme
accepts a recipe sentence and finds a basic action           into to two or more, then try to check the
corresponding to it from the lexicon. One of the            equivalence of both expressions.
biggest difficulties in developing this module is
                                                         • Inconsistency in case fillers
that linguistic expressions in a recipe may differ
from those in the lexicon. So we have to consider           Verbs in a recipe and the lexicon agree, but
a flexible matching algorithm between them.                  their case fillers are different. For instance,
   To construct Action Matcher, we refer to the             in Figure 5 (b), the verb “      (sprinkle)” is
verbal phrases classified in (b2) in Table 2. Note           the same, but the accusative case fillers “
that the linguistic expressions of these verbal                (chili)” and “ (salt)” are different. In this
phrases are inconsistent with the expressions in the        case, we can regard both as representing the
lexicon. We examined the major causes of incon-             same action: to sprinkle a kind of condiment.
sistency for these verbal phrases. In this paper, we        In this case, the lexicon of ingredients (see
will report the result of our analysis, and suggest         4.2.2) would be helpful for matching. That
some possible ways to find the equivalent action             is, if both         (chili) and     (salt) have
even when the linguistic expressions in a recipe            the same feature “kind=condiment” in the
and the lexicon are different. The realization of           lexicon of ingredients, we can judge that
Action Matcher still remains as future work.                the phrase “        / /       (sprinkle chili)”
   Figure 5 shows some examples of observed in-             corresponds to the basic action “ / /
consistency in linguistic expressions. In Figure 5,         (sprinkle salt)”.
the left hand side represents verbal phrases in
recipes, while the right hand side represents ex-        • Inconsistency in verbs
pressions in the lexicon of cooking actions. A              Disagreement between verbs in a recipe and
slash indicates word segmentation. Causes of in-            the lexicon is one of the major causes of in-
consistency in linguistic expressions are classified         consistency. See Figure 5 (c), for instance.
                  Expressions in Recipes                                          Expressions in Lexicon
 (a)           /                 · · ·break (egg)                                                  · · ·break (egg)
     (divide) (loosen)                                                    (break)
 (b)          /      /           · · ·sprinkle chili                          /       /         · · ·sprinkle salt
      (chili) (ACC) (sprinkle)                                         (salt) (ACC) (sprinkle)
 (c)                 /     /     · · ·make (shellfish)                               /     /     · · ·dip it into
     (Spewing sand) (ACC) (do)        spew out sand                    (salt water) (LOC) (dip)      salt water

                                  Figure 5: Inconsistency in Linguistic Expressions

       These two phrases represent the same ac-                    Elisabeth Andre and Thomas Rist. 1996. Coping
       tion 5 , but the linguistic expressions are to-                with temporal constraints in multimedia presenta-
                                                                      tion planning. In Proceedings of the National Con-
       tally different.
                                                                      ference on Artificial Intelligence, pages 142–147.
       In this case, the matching between them is                  Yoko Atsuta. 2004. How to cut vegetables (in
       rather difficult. One solution would be to de-                              u
                                                                     Japanese). Syˆ eisha.
       scribe all equivalent expressions for each ac-
       tion in the lexicon. Since it is not realistic to           Bob Coyne and Richard Sproat. 2001. WordsEye: An
                                                                     automatic text-to-scene conversion system. In Pro-
       list equivalent expressions exhaustively, how-                ceedings of the SIGGRAPH, pages 487–496.
       ever, we want to automatically collect pairs
       of equivalent expressions from a large recipe               Yoshiko Fujino. 2003. New Fundamental Cooking (in
                                                                     Japanese). SS Communications.
                                                                   Eri Hayashi, Suguru Yoshioka, and Satoshi Tojo. 2003.
6 Conclusion                                                          Automatic generation of event structure for Japanese
                                                                      cooking recipes (in Japanese). Journal of Natural
                                                                      Language Processing, 10(2):3–17.
In this paper, we have described the basic idea for
a system to generate animations for cooking ac-                    Robin F. Karlin. 1988. Defining the semantics of ver-
tions in recipes. Although the system is not yet                     bal modifiers in the domain of cooking tasks. In
complete and much work still remains to be done,                     Proceedings of the Annual Meeting of the Associ-
                                                                     ation for Computational Linguistics, pages 61–67.
the main contribution of this paper is to show the
direction for improving the scalability of the sys-                Tomohide Shibata, Daisuke Kawahara, Masashi
tem. First, we designed a lexicon of cooking ac-                     Okamoto, Sadao Kurohashi, and Toyoaki Nishida.
                                                                     2003. Structural analysis of instruction utterances.
tions including information about action plans and                   In Proceedings of the Seventh International Con-
ingredient requirements, which are needed to gen-                    ference on Knowledge-Based Intelligent Information
erate the appropriate cooking animations. We also                    and Engineering Systems (KES2003), pages 1054–
showed that our lexicon covers most of the cook-                     1061.
ing actions appearing in recipes. Furthermore, we                  Junko Takashiro and Satomi Kenmizaki.     2004.
analyzed the recipe corpus and investigated how                      Standard Cooking: Fundamentals of Cooking (in
to match actions in a recipe to the corresponding                                 o
                                                                     Japanese). Shˆ gakukan.
basic action in the lexicon, even when they have                   Stuart G. Towns, Charles B. Callaway, and James C.
different linguistic expressions. Such a flexible                      Lester. 1998. Generating coordinated natural lan-
matching method would also increase the scala-                        guage and 3D animations for complex spatial expla-
bility of the system.                                                 nations. In Proceedings of the National Conference
                                                                      on Artificial Intelligence, pages 112–119.
                                                                   Hideki Uematsu, Akira Shimazu, and Manabu Oku-
References                                                           mura. 2001. Generation of 3D CG animations
                                                                     from recipe sentences. In Proceedings of the Nat-
Hisahiro Adachi. 1997. GCD: A generation method                      ural Language Processing Pacific Rim Symposium,
  of cooking definitions based on similarity between                  pages 461–466.
  a couple of recipes. In Proceedings of the Natural
  Language Processing Pacific Rim Symposium, pages                  Bonnie Lynn Webber and Barbara Di Eugenio. 1990.
  135–140.                                                           Free adjuncts in natural language instructions. In
                                                                     Proceedings of the International Conference on
     Note that it is required to dip shellfish into salt water in     Computational Linguistics, pages 395–400.
order to make it spew out sand.

To top