Document Sample
Intensional_Context-Free_Grammar Powered By Docstoc
					Intensional Context-
Free Grammar
Thesis Topic
   Natural language processing (NLP) is the use of
    computers to understand human languages
   Generative grammar is the use of rules to
    construct sentences
   Intensional context-free grammar (ICG) is a
    generative grammar born out of the intensional
    programming paradigm
     based   on intensional logic
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
     Future   Work
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
     Future   Work
Natural Language Processing
   Subfield of Artificial Intelligence (AI)
       Typically seen as one part of knowledge representation
   Can also be seen as a combination of computer science
    and linguistics
       Computational Linguistics
   Applications
       English as a command language
       Database queries
       Translation systems
       Speech recognition systems
   Human language divides into five levels
    1.   Phonology
            How sounds are used in language
    2.   Morphology
            Word formation
    3.   Syntax
            Sentence formation
    4.   Semantics
            Sentence meaning
    5.   Pragmatics
            Use of language in context
   Generative grammars are mostly concerned with
     Morphology    and semantics to some extent as well
   Words can be grouped together into syntactic
     Classifiedby type of meaning, how they can be
      altered and where they can occur
   They fall into two types
     Lexical   and non-lexical
Lexical Categories
   Noun (N) – Typically names entities
        Can be inflected for number and possession
        Harry, boy, wheat, painting
   Verb (V) – Designates actions, sensations and states
        Can be inflected for tense
        Arrive, discuss, melt, hear
   Adjective (A) – designates a property of a noun
        Good, tall, old
   Preposition (P) – links nouns to other words
        To, in, on, through, at, by
   Adverb (Adv) – Names properties of verbs
        Silently, slowly, now
Non-lexical Categories
   Determiner (D) – specifies a N
       The, a, these, every, my
   Auxiliary (Aux) – specifies a V
       Will, can, may, could
   Degree (Deg) – specifies a P or an A
       Too, so, very, more, often
   Qualifier (Qual) – modifies a N or V
       Always, perhaps, often, never
   Conjunction (Con) – joins two or more categories of the
    same type
       And, but, or
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
   Future Work
Generative Grammar
   The idea of grammar has been around for centuries in
    one form or another.
   It wasn’t until the middle of the last century that the idea
    of formal grammar took hold.
   At that time Noam Chomsky and others had the
    revolutionary idea that some part of language learning in
    humans is innate.
       Languages are infinite
       Language acquisition is relatively simple
       Evidence that the brain is structured for language
   Sentences are constructed using rules
       Not listed
Phrases and Sentences
   Sentences have a hierarchical design
   Words are grouped together into structural units
     Phrase   structures or constituent structures
   Phrase structures are based on lexical
   Typically have a head, specifier and a
    XP -> (Spec) X (Comp)
Phrase Structure
   The head is the word around which the phrase is built (lexical)
        NP, VP, PP, AP, and AdvP
   Specifiers are words that specify the head
      D, Aux, Deg, Qual
      NP -> A dog
      VP -> will run
      AP - > quite certain
      PP -> almost in
   Compliments are phrases themselves that provide information
    about the head
        NP -> the books [about the war]PP
        VP -> never eat [a hamburger]NP
Example Grammar
S -> NP VP          Example sentence
NP -> D N                             S
                           NP                 VP
VP -> V NP
                      D         N         V          NP
D -> the | a
                                               D           N
N -> dog | cat
V -> chased | saw    the        dog   saw      the        cat
Classification of Generative
   In his pursuit of a generative grammar for
    natural languages Chomsky defined different
    grammar types
   This created a hierarchy of computability based
    on complexity of computation
   Chomsky’s Hierarchy
     Type   0 – recursively enumerable (r.e.)
     Type   1 – context-sensitive (CSG)
     Type   2 – context-free (CFG)
     Type   3 – regular (RG)
Where do Natural Languages Fall?
   Regular grammars were quickly ruled out
   Chomsky contended that natural language could
    not be generated with a context-free grammar
     some   context-sensitivity was necessary.
   Some mathematical proofs to this effect were
   The first attempt at a grammar of natural
    language was the Transformational Grammar
    proposed by Chomsky
Transformational Grammar (TG)
   there exist two types of structure, deep structure (d-
    structure) and surface structure (s-structure)
   The d-structure is generated by context-free rules and
    contains the bulk of the meaning of the sentence
   The surface structure is reached by a series of
    transformations which account for the morphology and
    word order of the sentence.
   The transformations alter the d-structures in ways that
    cannot be accounted for by CFG alone
   Consider the sentence (passive):
    (1) A book was given to the professor.
   In TG, (1) is the s-structure of the given
    sentence and is actually of the form:
    (2) A book was given _ to the professor
   This is the result of the application of an
    NP-movement on the d-structure:
    (3) _ give a book to the professor.
Example TG
   The production rules necessary to generate this d-structure are
        S  NP I VP
        NP  D N | e
        VP  V NP PP
        PP  P NP
        I  past
        D  ‘a’ | ‘the’
        N  ‘book’ | ‘professor’
        V  ‘give’
        P  ‘to’

    NP    I                    VP

    e    past   V              NP                   PP

                ‘give’   D          N        P               NP

                         ‘a’        ‘book’   ‘to’        D        N

                                                     ‘the’   ‘professor’

        NP             I                      VP

   D          N        ‘was’         V        NP          PP

  ‘a’        ‘book’
                                    ‘given’   e     P                 NP

                                                   ‘to’           D         N
                      NP movement

                                                          ‘the’       ‘professor’
Problems with TG
   Empty categories, non-word lexical entries, and d-
    structures aren’t sentences
   TGs are weakly equivalent to the r.e. languages.
       anything that is computable can be modeled by a TG
       TG is too complex for use as a natural language generator.
   Transformational grammars vs. non-transformational
       restrict the transformational grammar in some way
       look back at the context-free grammars
Restricting TG
   Wasow suggests constraining the language of
    transformational grammar in such a way to make it a
    context-sensitive grammar
   Savitch shows that any context-sensitive grammar can
    generate for each recursively enumerable language one
    just as complex.
       Thus, CSG is also a poor choice for a formal model of language
   Lead to a string of more restrictive TGs
       The Standard Theory, The Revised, Extended Standard Theory,
        Realistic Transformational Grammar, Government and Binding
        Theory, and the Minimalist Program
Back to CFGs
   Harman, Pullum and Gazdar suggest that previous
    arguments against the use of CFGs have been
    misleading and mathematically unsound
       examples are given of CFGs that generate subsets of English
   A number of papers then appeared with examples in
    different languages refuting the use of pure CFGs.
   in Bresnan et al, they show that certain languages while
    possibly weakly equivalent to CFLs, are not strongly
   In others, Shieber, Higginbotham and Culy, they give
    examples of languages that are neither weakly nor
    strongly equivalent to CFLs.
Non-Transformational Grammars
   Large majority of the constructs of languages can be
    generated through the use of CFGs.
   The non-transformational grammars
       Lexicalization
       Subcategorization
   Examples
       Head-driven Phrase Structure Grammar
       Tree Adjoining Grammar
       Indexed Grammar
       Lexical-Functional Grammar
Lexical-Functional Grammar (LFG)
   ‘Lexical’ - LFG has a stronger lexicalization than
    a  lot of the work of the grammar is done at the word
   ‘Functional’ - the role of grammatical functions is
     propose   a separate functional structure (f-structure)
   ‘Grammar’ - a model of generative grammar that
    is an extension of CFG
   There are three important structures in LFG
     c-structure (constituent structure,   CFG)
     f-structure (functional structure)
     θ-structure ( thematic roles)
          agent, theme, location
   Furthermore, there exists two types of mappings
     f-description which allows for the mapping between
      c-structure and f-structure
     the a-structure which is a map between the f-
      structure and θ-structure.
  Example LFG
     Consider sentence (1) again
            A book was given to the professor
     Here are the rules, f-descriptions and lexical entries
S  NP                   I     VP                            (↑TENSE) = PAST
      SUBJ                          was      I
                                               a        D    (↑DEF) = -
NP  D             N                                         (↑NUM) = SG
                
                                               the      D    (↑DEF) = +
VP  V             NP              PP          book     N    (↑PRED) = ‘book’
           OBJ         OBLgoal                  (↑NUM) = SG
                                               professor N   (↑PRED) = ‘professor’
VP  V               PP                                      (↑NUM) = SG
            OBLgoal 
                                               given    V    (↑PRED) = ‘give
PP  P              NP                                               (↑OBLgoal)’
             OBJ 
                                               to      P     (↑PCASE) = OBLgoal

               NP               I                             VP
             SUBJ                                     

         D           N          was          V                                      PP
                    TENSE  PAST                                 OBLGoal 

     a            book                      given                     P                        NP
 DEF    PRED 'book '                                                            OBJ 
                                       PRED  ' give  '
 NUM  SG NUM  SG

                                                                     to               D                 N
                                                               PCASE OBLGoal                     

                                                                                     the            professor
                                                                                   DEF    PRED ' professor '
                                                                                                   NUM  SG
   Also, called the attribute-value matrix (AVM),
       is a nested matrix of grammatical functions (attributes) and there values

                                        DEF                        
               SUBJ                     PRED ' book '               
                                                                    
               TENSE                        PAST                      
               PRED
                           ' give   SUBJ   OBLGoal OBJ          '
               OBJ                                                    
                              PCASE          OBLGoal                
                                                                     
              OBLGoal                    DEF                      
                               OBJ       PRED ' professor' 
                                                                     
                                                                 
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
     Future   Work
Intensional Logic
   Intensional logic was motivated initially as a formal
    description of natural language meaning.
       Scott and Montague who looked to formalize the semantics of
        It was later applied by Carnap to assign meaning to sentences
        based on implicit contextual information
   The truth assignment of a sentence is dependent on the
    context or possible world
   Typically, this context is not stated explicitly but is
    implied by the world in which the statement is uttered
Intension and Extension
   The statement itself is defined as the intension
   the interpretation of that statement in the given context is
    defined as the extension of the statement.
   The extensions of some intensional statement can
    depend on any number of contexts
       time, space, culture, audience, etc.
   As an example consider the expression:
    (4) It is 12 degrees Celsius.
   The truth of this statement depends on at least two
       the time and place in which it was uttered
More Intensional Logic
   Different branches of intensional logic
        modal logic, epistemic logic (knowledge and belief), deontic logic
         (obligation and permission), tense logic and conditional logic.
   It seems a natural fit that languages use intensional logic
        most contextual information goes unstated in our every day life
        yet the meaning of statements are usually perfectly clear.
   Most languages even have context switching operators without
    actually calling them such
        ‘yesterday’ and ‘today’ for temporal switching
        ‘there’ and ‘here’ for spatial switching
   Since its inception though, intensional logic has been used more for
    other purposes
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
     Future   Work
Intensional Programming
   Intensional programming is a programming
    paradigm based on intensional logic
   The idea behind intensional programming is the
    use of context in any and all aspects of the
   Many examples of its use
     functional programming
     software version control
     web authoring
     scripting
Functional Programming
   The first programming language developed based on the
    principles of intensional logic is called Lucid.
   The creator of this language, William Wadge, wanted a
    programming language that avoided knowledge of the
    internal make-up of the system on which it would run.
   It was seen that hiding such information from the
    programmer is similar to the hidden details about context
    that are inherent to natural language use
   Lucid is an example of a functional-intensional language
        consists of functions which operate on streams of data, which are the
   A Lucid intension, x, is a value which varies over time
        the one dimension (context) of the Lucid environment
   So, the extensions of x are the infinite values that x takes on at each
    time point t.
   The time points in Lucid are implied
   but there are three context-switching operators available to the
        first, next and fby.
   Lucid stores previously calculated results (extensions) in a
    warehouse, known as warehousing.
Extensions of Lucid
   Applying intensional programming to other programming
    paradigms is continued and Lucid is extended in several
       spreadsheet programming, databases, real-time systems,
        logic programming (Chronolog).
   Other extensions are discussed in more detail in this
   Lucid is also extended to contain both a declarative and
    an imperative part and it is called Granular Lucid (GLU).
   Two important ideas that come out of the work of GLU
     multidimensional versioning
     versions as values
Software Versioning
   One particularly interesting use of intensional
     version   control tools in software development.
   In 1993, Plaice and Wadge applied intensional
    contexts to versioning
     differentpossible versions of software components as
      possible worlds.
   A complete system is formed by taking the most
    relevant version of each component.
Version Algebra
   To do this they developed a version algebra,
     partially
             ordered by an operation called refinement,
      denoted by .
   V  W read as W refines V
     means  that version W of a particular component is
      an extension of version V
   The simplest version of any component is called
    the vanilla version and is denoted by 
   Allows for joins (+) of versions and subversions
Best-fit Algorithm
   The complete version V of a system is found by
    selecting the appropriate component versions.
   best-fit algorithm - the most relevant version of
    each component is selected
   Thus, suppose we are looking for the
    Keir%apple%fast version of a component
     exists
           in the versions Keir, Keir%apple, Keir%fast,
      apple%fast, Maria%apple, fast, and .
   In this case the most relevant version would be
    the Keir%apple version.
Web Authoring
   The version control system becomes important in the
    intensional programming world.
   One of the first uses of the version space phenomenon
    comes in the form of web authoring languages
   Successive web authoring programs
       Intensional Hypertext Markup Language (IHTML), IHTML 2
        and IHTML 3, which all extend HTML with intensions.
   This is done by revising the version space system and
    best-fit algorithms to account for dimension (context)
    labels and the use of dimensions as values themselves.
   IHTML allows authors to define a whole indexed
    family of HTML variants using a single source
   The intension is the family of HTML pages while
    each individual page serves as an extension
   So, authors can provide multiple sources for the
    same page where each source is labeled with a
    different version.
   The version dimensions can be attributed to any
    of the markup elements of traditional HTML.
Versioning in IHTML
   Explicit dimension identifiers separated from the value of
    that dimension with the notation (:).
   So, for example we can have the version
   Another added element of IHTML is the use of
    transversion links
       links that are used to change the context of the current version
        using vmod
   Example, suppose your current context is
       <a href=page1 vmod=”language:french”>
   improves upon IHTML in a number of
     implementation   on the server-side and overall
      efficiency of the system
     addition of dimensions as values
     change in the best-fit algorithm

   IHTML 2 drops the notion of a subversion
   Swoboda extends IHTML 1 and 2 by letting
    dimension identifiers be nested to an arbitrary
    depth and by adding imperative structures
   ISE (imperative scripting language)
     inthe spirit of Perl but uses versioning for all of its
      identifiers, including files, variables, and functions
   allows the values of dimensions to be version
    expressions themselves
           d1 : i1  d 2 : V2    d1 : i1  d1 : d 2 : V2  .
Intensional Markup Language (IML)

   Wadge and schraefel added IML to ISE as
    a front end
     to get back to the simplicity of markup while
      still maintaining the power of ISE.
 IML uses Groff macros to extend HTML
 IML is translated into ISE which in turn is
  translated into HTML readable by your
   Natural Language Processing
     Linguistics
   Generative Grammars
     Transformational grammar
     Lexical-functional grammar
   Intensional Logic
   Intensional Programming
   Intensional Context-Free Grammar
     Future   Work
Intensional Generative Grammar

 The one area that has seen little use of
  intensional programming is in generative
 Considering the motivation for intensional
  logic in the first place, this seems remiss.
 But, there has been some work in this vain
Sentence Generators
   two examples of sentence generation using the
    intensional paradigm
   In both cases, the generator is built on ISE and
    thus is web specific.
   Both use a small grammar and lexicon and
    construct French sentences by an informal
   sentence construction here is not via generative
    grammar methods, that is, there are no CFG-like
Intensional Context-Free Grammar
   Although, the intensional logic paradigm grew out of
    research in natural language semantics not much work
    has been done in NLP with intensional programming.
   Intensional context-free grammar (ICFG) consists of two
       a constituent structure (c-structure), composed of a CFG with
        two types of production rules,
            tagged rules and context switching rules
       a warehouse structure (w-structure) which stores the possible
        world view as a version space.
Future Work
   My intention is to formally define the grammar
    underlying the intensional programming
     denotational   and operational semantics
   I also plan on developing a practical grammar
    for use in NLP,
     English sentence generator
     Might use AVMs from LFG as     version space
   I may also develop an intensional prolog
    interpreter to be used for this grammar