A Hierarchy of Languages and Automata by langkunxg


									CS 3240 – Chapter 11
   They may not halt on every possible input!
     And not just because the creator of a specific TM was a

   This is related to the major
    mathematical/computational discovery of the 20th
     There are propositions that cannot be decided (“proven”)
   A question is decidable if there is a TM that always
    halts and answers “yes” or “no” for each possible
     The TM therefore constitutes an algorithm
   A language is decidable if there is a TM that always
    halts and answers “accept” or “reject” whenever an
    input string is in the language or not
     aka “Turing acceptable”
   A function is computable if there is a TM that
    always halts with the appropriate output for
    each possible input in the function domain
     aka “Total function”
 Let g(x,y) be some computable function
 Let f(x) =
     the smallest p where g(x, p) = 1, or
     0, if such a p does not exist
 “Pseudo-algorithm” for f(x):
    m = 0;
    while (g(x, m) != 1)
    cout << m;
   If there is no m for a given g and x, then we hang!
   3 possibilities in general when a TM processes
    an input string:
     Accepts (goes to an accepting halt state)
     Rejects (e.g., crashes, or gives a “no” answer)
     Hangs (infinite loop)
   A language for which there is a TM that
    always halts for and accepts strings in the
    language is recognizable
     “It knows one when it sees one” :-)
     But it may hang on strings not in the language
   A function that is defined for only some of its
    domain elements is a partial function
     It may hang (or “blow up”) on some inputs
     e.g., divide by zero
   This is the computational analogue to a
    recognizable language
     you’ll always get an answer with a valid input
     it may hang on invalid input
Language                    Machine              Grammar
Regular                     Finite Automaton     Regular Expression,
                                                 Regular Grammar
Context-Free                Pushdown Automaton   Context-Free
Recursively                 Turing Machine       Unrestricted Phrase-
Enumerable                                       Structure Grammar

               CS 3240 - Introduction                                   9
A language L over the alphabet  is called recursively
enumerable (aka “recognizable”) if there is a TM T that accepts
every word in L and either rejects (crashes) or loops forever for
every word in the language L', the complement of L:
                   accept(T) = L
                   reject(T) + loop(T) = L'

                      recursively enumerable (r.e.) = recognizable
   We just saw one (slide 5)
   Let g(x,y) be some computable function
   Let f(x) =
        the smallest p where g(x, p) = 1, or
        0, if such a p does not exist
   Let Lg be the set of functions, f, corresponding to
    all computable functions g, as explained above
     Such functions can be encoded as strings, and are
      therefore countable
     Lg can then be seen as a language (a set of strings)
A language L over the alphabet  is called recursive (aka
“decidable”) if there is a TM T that accepts every word in L and
rejects every word in L'; that is,
                      accept(T) = L
                       reject(T) = L'

                      A TM that accepts a recursive set is a
                      model of an algorithm. It always halts.
                      recursive = decidable
   There exist languages that are r.e. but not recursive
     We just saw one
   There exist languages that aren't even r.e.!
     (You'll see one soon)
     All are “contrived”
   Languages generated by grammars are r.e. or
   All r.e. languages are closed under union,
    intersection, concatenation, and Kleene*
     Everything but complement!
   Recursive languages are also closed under
   Also: If L and L' are r.e., then L is recursive
   Here come the Proofs…
   Let M be a machine that decides a recursive
    language, L
   Form the machine M' by inverting the
    acceptability output of M
     Goes to a reject state instead
   Then M' decides L'
     So L' is recursive
   Suppose L and L' are both r.e.
   Let M recognize L, and M' recognize L'
     M may hang on elements of L', but M' doesn't
   Form a new machine, M* that calls M and M'
    in parallel (non-deterministically)
       If M accepts w, so does M*
       If M' accepts w, reject w
       There are no other possibilities! (No hanging)
       Therefore, L is decidable/recursive, by definition
   TMs can recognize/accept strings from certain
    languages and/or compute functions

   If there is a TM, M, that accepts a language, L, and M
    always halts, then L is recursive

   If there is no such M for L, but there is instead a
    machine M that accepts every string in L, but M may
    hang on strings not in L, then L is recursively
   The complement of a recursive language is
     In fact, recursive languages are closed under all
      operations, like regular languages are
   r.e. languages are closed under intersection
   The complement of a r.e. language may not
    be r.e.
     But if it is, then both languages are actually
A Timely Interlude
   The real numbers in (0,1) are uncountable
     They cannot be mapped in a 1-to-1 fashion to the
     counting numbers
   Proof:
     Assume they can be: r0, r1, r2, …
     Arrange their digits in a table a[ ][ ]
      ▪ each row, a[i], contains the digits or ri
     The diagonal sequence (a[n][n]+1) mod 10,
     representing a valid real number, is not in the
     table! Contradiction!
   The power set of the natural numbers, N
     the set of all subsets (2N)
   Suppose it is countable
     The we can enumerate the sets: p0, p1, p2, …
   Now consider the set T = {i | i ∉ f(i) = pi }
     Certainly T is a set of integers, so T ∈ P(N)
      ▪ Call it pk
     Question: Is k in T = pk?
   The power set of any countably infinite set is
   Much bigger than a countable set!
     The number of countable sets is negligible
     compared to the number of uncountable sets
   Well?
   A language over an alphabet Σ is a subset of
     the latter being an infinite set
   The set of all languages over Σ is therefore the
    power set of Σ* (2∑*)
     which we just showed is uncountable
   So… the number of languages over any finite
    alphabet is uncountable
   The # of TMs are countable, but the # of
    languages is not
   Therefore, some languages cannot be
    recognized by a TM
     There aren't enough TMs to go around!
     Just like there are more reals than integers
   So, non-r.e. languages must exist!
   Just take the complement of any r.e. language
    that is not recursive
   Example (page 279):
     Consider all TMs, Mi with alphabet Σ={a}.
     Let X = {ai: ai ∈ L(Mi)}
      ▪ This is r.e., but not recursive
      ▪ Because we can construct a TM that carries out the
        computation, but it may not halt when ai ∉ L(Mi)
     Then X’ must be non-r.e.!
   Suppose X’ is r.e.
     Then there is a TM, Mk, that recognizes it
   Now ask the question, is ak ∈ X’ ?
   If it is, this means ak ∉ L(Mk) = X’, by definition
     Contradiction!
   If it isn’t, then this means that ak ∈ L(Mk) = X’
     Again, by definition
     Again, a contradiction!
   ⇒ X’ is not r.e.
   What we have just described is the
    membership problem:
     “Given a r.e. language, L, and a string, w, is w in
   We have just shown that the membership
    problem is, in general, undecidable
Language                   Machine              Grammar
Regular                    Finite Automaton     Regular Expression,
                                                Regular Grammar
Context-Free               Pushdown Automaton   Context-Free
Recursively Enumerable Turing Machine           Unrestricted

               CS 3240 - Introduction                                 29
   Left-hand side of the rule is a concatenation
    of one or more symbols
     There must be at least one variable on the left
      ▪ λ is not allowed on the left
   Any string is allowed on the right, including λ
   aka “Type 0” Grammar
S → aAbc | abc | λ                          S ⇒aAbc
Ab → bA                                     ⇒abAc
Ac → Bbcc                                   ⇒abBbcc
bB → Bb                                     ⇒aBbbcc
aB → aa | aaA                               ⇒aabbcc

An A is created as the left-most variable. It travels to the right until it finds a
'c', then creates a new b and c, and becomes a B. The B moves back to create
the extra needed a. The last rule allows the option to do it all over again (by
introducing another A). This is similar to what the Turing machine for the
same language does, except the TM marks instead of generates the letters.
 We have already seen            S → XYS | λ
  CFGs for this                    XY → YX
 This unrestricted                YX → XY
  grammar:                         X→a
     Introduces X’s and Y’s       Y→b
     Allows moving X’s and
      Y’s arbitrarily by
      swapping them
   Cannot use the decidability algorithms for
     e.g., CYK algorithm does not apply
   No “Normal Form”
   Non-null productions may create shorter
   Terminals can disappear!
   For every r.e. language, there is an
    unrestricted grammar that generates it
   For every unrestricted grammar, there is a TM
    that recognizes its language
     It may or may not decide it
   We will not prove this
     but the book does
   Like Unrestricted Grammars except:
     Right-hand side must be no shorter than left-hand side
      ▪ So strings never shrink
     Hence the name “non-contracting” or “monotonic”
     Cannot contain λ anywhere
   Context-sensitive languages don't need unlimited
     Since intermediate sentential forms never shrink, the
      largest memory requirement is proportional to |w|
     Accepting machine: Linear Bounded Automaton
S → aAbc | abc
Ab → bA
Ac → Bbcc
bB → Bb
aB → aa | aaA
   Every context-sensitive language is recursive
     there is a TM that accepts it (i.e., always halts)
   But not all recursive languages are context-
       So Context Sensitive languages form a proper
        subset of Recursive Languages
       Just like Recursive languages are a proper subset of
        Recursively enumerable languages
Type 0 grammar

Type 1 grammar

Type 2 grammar

Type 3 grammar
Language                     Grammar                         Machine
Recursively Enumerable       Unrestricted Phrase             TM (that may not halt)
(“Turing Recognizable”)      Structure (Type 0)
Recursive                               "                    TM (that always halts)
(“Turing Decidable”)
Context-Sensitive            Context Sensitive (Type 1)      Linear-bounded automata
                             (monotonic/non-                 (bounded TM)
Context Free                 Context Free (Type 2)           PDA

Deterministic Context Free              "                    DPDA

Regular                      Regular (Type 3)                FA
                             (Right-linear or Left-linear)

To top