Introduction to Computability Theory by malj

VIEWS: 5 PAGES: 50

									    Introduction to Computability
               Theory
        Lecture11: The Halting
               Problem
             Prof. Amos Israeli




1
The Halting Problem
In this lecture we present an undecidable
    language.
The language that we prove to be undecidable is
   a very natural language namely the language
   consisting of pairs of the form M , w where M
   is a TM accepting string w:
                                         
        ATM  M , w  M is a TM accepting w

2
The Halting Problem
Since this language requires to decide whether
   the computation of TM M halts on input w, it
   is often called The Halting Problem.


Theorem
The halting problem is Turing Recognizable.


3
Proof
Consider a TM U that gets a pair M , w as input
  and simulates the run of M on input w. If M
  accepts or rejects so does U. Otherwise, U
  loops.
Note: U recognizes ATM ,since it accepts any pair
  M , w  L , that is: any pair in which M accepts
  input w.

4
Simulating an Input TM
On the previous lecture, we detailed the
  simulation of a DFA by a TM.
Simulating one TM by another, using the
   encoding of the first TM is a very similar
   process. In the next slide we review the main
   characteristics of TM N simulating TM M,
   using M’s encoding <M>.

5
Simulating an Input TM
TM N works as follows:
1. Mark M’s initial state and w’s initial symbol as
   the “current state” and “current head
   location”.
2. Look for M’s next transition on the
   description of its transition function.
3. Execute M’s transition.
6
Simulating an Input TM
4. Move M’s “current state” and “current head
   location” to their new places.
5. If M’s new state is a deciding state decide
   according to the state, otherwise – repeat
   stages 2-5.



7
Diagonalization
Our goal in this lecture is to prove that ATM is
  not decidable. The proof uses a common
  mathematical technique known as
  Diagonalization.
Diagonalization was first used by Cantor when
   he found a way to distinguish between
   several types of infinite sets.

8
Cardinality
Cantor dealt with questions like:
How many natural numbers are there? Infinity!
How many real numbers are there? Infinity!
Does the amount of natural numbers equal to
  the amount of real numbers?
How is the size of infinite sets measured?

9
Cardinality
Cantor’s answer to these question was the
  notion of Cardinality.
The cardinality of a set is a property marking its
   size.
Two sets has the same cardinality if there is a
  correspondence between their elements


10
Intuitive Notion of Correspondence
At this point of the lecture, think about a
   correspondence between sets A and B as 2
   lists: A list of A’s elements and in parallel a
   list of B’s elements. These 2 lists are
   juxtaposed so that each element of A
   corresponds to a unique element of B.


11
Example
 A   ,2,3,4 B  2,4,6,8
                               A 1 2 3 4
     1
                               B 2 4 6 8


Clearly, the cardinality of A is equal to the
   cardinality of B.
How about the cardinality of infinite sets?



12
Example
How about the cardinality of infinite sets?
Is the cardinality of natural numbers larger than
    the cardinality of even natural numbers?
Intuitively, the cardinality of any set should be
   larger that the cardinality of any of its proper
   subsets. Alas, our intuition of sets is driven
   by our daily experience with finite sets.
13
Example
So let us try to create a correspondence
   between the natural numbers the even
   natural numbers?
     N   ,2,3,4,...,n,...
         1                     N    1 2 3 ... n ...
EN  2,4,6,8,...,2n,...      EN   2 4 6 ... 2n ...

Indeed f n  2n defines the wanted
   correspondence between the 2 sets.
14
Example
     N   ,2,3,4,...,n,...
         1                     N    1 2 3 ... n ...
EN  2,4,6,8,...,2n,...      EN   2 4 6 ... 2n ...

So the cardinality of N is equal to the
   cardinality of EN.




15
Countable Sets
This last example suggests the notion of
   Countable Sets:
A set A is countable if it is either finite or its
   cardinality is equal to the cardinality of N.
A cool way of looking at countable sets is:
   “A set is countable if a list of its elements can
   be created”.
16
Countable Sets
“A set is countable if a list of its elements can be
   created”.
Note: This list does not have to be finite, but for
  each natural number i, one should be able to
  specify the i-th element on the list.
For example, for the set EN the i-th element on
   the list is 2i .
17
Countable Sets
We just proved that EN, the set of even natural
  numbers is countable. What about the set of
  rational numbers?
Is the set Q of rational numbers countable?
Can its elements be listed?



18
The set of Rationals is Countable
Theorem
The set of rational numbers is countable.
Proof
In order to prove this theorem we have to show
   how a complete list of the rational numbers
   can be formed.


19
The set of Rationals is Countable
Recall that each natural number is defined by a
   pair of natural numbers.
 One way to look     1 / 1 1 / 2 1 / 3 1 / 4 1 / 5 ………
 at the Rationals    2 / 1 2 / 2 2 / 3 2 / 4 2 / 5 ………
is by listing them   3 / 1 3 / 2 3 / 3 3 / 4 3 / 5 ………
                     4 / 1 4 / 2 4 / 3 4 / 4 4 / 5 ………
in an infinite
                     5 / 1 5 / 2 5 / 3 5 / 4 5 / 5 ………
   Rectangle.
                     ………………………………………………...
20
The set of Rationals is Countable
How can we form a list including all these
  numbers?
If we first list    1 / 1 1 / 2 1 / 3 1 / 4 1 / 5 ………
The first row –     2 / 1 2 / 2 2 / 3 2 / 4 2 / 5 ………
We will never       3 / 1 3 / 2 3 / 3 3 / 4 3 / 5 ………
                    4 / 1 4 / 2 4 / 3 4 / 4 4 / 5 ………
reach the second.   5 / 1 5 / 2 5 / 3 5 / 4 5 / 5 ………
                    ………………………………………………...
21
The set of Rationals is Countable
     1/1 1/ 2 1/ 3 1/ 4 1/ 5
     2 /1 2 / 2 2 / 3 2 / 4 2 / 5
     3 /1 3 / 2 3 / 3 3 / 4 3 / 5
     4 /1 4 / 2 4 / 3 4 / 4 4 / 5
     5 /1 5 / 2 5 / 3 5 / 4 5 / 5

     One way to do it is to start from
     the upper left corner,
     and continue in this fashion
22
The set of Rationals is Countable
Note that some rational numbers appear more
  than once. For example: all numbers on the
  main diagonal are equal to 1, so this list is
  not final.
In order to compute the actual place of a given
    rational, we need to erase all duplicates, but
    this is a technicality…

23
So perhaps all sets are countable
Can you think of any infinite set whose elements
  cannot be listed in one after the other?
Well, there are many:
Theorem
The set of infinite binary sequences is not
   countable.


24
Uncountable Sets
Assume that there exists a list of all binary
   sequences. Such a list may look like this:
     1 0 1 1 0          ………
     1 1 0 0 1          ………
     0 0 0 0 1          ………
     1 1 1 0 1          ………
     1 0 0 0 1          ………
     …………………….................
25
Uncountable Sets
But can you be sure that all sequences are in
   this list?
In fact, There exist    1 0 1 1 0          ………
    many sequences      1 1 0 0 1          ………
    that are not on     0 0 0 0 1          ………
                        1 1 1 0 1          ………
    the list:                              ………
                        1 0 0 0 1
                        …………………….................
26
 Uncountable Sets
 Consider for example S=0,0,1,1,0,… . The
    sequence S is formed so that
S1  1st elt. Of 1st seq.    1 0 1 1 0 ………
S2  2nd elt. Of 2nd seq.    1 1 0 0 1 ………

S3 
                                      0 0 0 0 1 ………
         3rd   elt. Of   3rd   seq.
                                      1 1 1 0 1 ………
 And so on …                          1 0 0 0 1 ………
                                      ……………………...........
27
Uncountable Sets
In general: The i-th element of S, S i  differs
    from the i-th element of the i-th sequence in
    the list. Note: the i-th element of the i-th
    sequence in the list is always the element on
    the diagonal.
Can the sequence S appear on the list?


28
Uncountable Sets
                                   Assume there
                                   exists an index j
 S1   1   0 1     1   0 ... ...
                                   such that S  S j
S2    1 1     0   0 1 ... ...
                                   In this case,
                                   S j  j  S j
 S3   0   0   0   0 1 ... ...
S4    1 1     1   0 1 ... ...
                                   But by definition:
                                    S j  S j  j
 S5 1 0 0 0 1 ... ...
 ... ... ... ... ... ... ... ...
                                   Contradiction!!
 S    0   0 1     1   0 ... ?


29
Uncountable Sets
For any sequence on the list, S j , the sequence
  S differs from S j by its element on the
   diagonal, that is: S  j   S j  j  , so S cannot
   be on the list.
For obvious reasons, this technique is called
   Diagonalization.


30
Uncountable Sets
We just used diagonalization to prove that the
  set of infinite binary sequences is
  uncountable.
Can a a similar proof for the set of real
  numbers?



31
Turing Unrecognizable Languages
Corollary
Some Languages are not Turing-recognizable.
Proof
For any (finite) alphabet,  , the set of (finite)
   strings  * , is countable. A list of all elements
   in  * is obtained by first listing strings of
   length 1, then 2, …, then n…
32
Proof (cont.)
The set of all TM-s is also countable because
   every TM, M , can be described by its
   encoding M , which is a string over  . So
   the set of TM-s corresponds to a subset of  *.
Note: Here we use the (unproven but correct)
  fact that the cardinality of a set is always not
  greater then the cardinality of any of its
  supersets.
33
Proof (cont.)
Since each TM recognizes exactly a single
   language, a list of all TM-s can be used as a
   list of all recognizable languages.
If we show that the set of languages over  is
    uncountable, we can deduce that at least a
    single language is not on the list, that is: it is
    not recognized by any TM.

34
Proof (cont.)
We have already seen that the set of infinite
  binary sequences is uncountable. If we form
  a correspondence between the set of
  languages and the set of infinite binary
  sequences we will show that the set of
  languages is uncountable.


35
Proof (cont.)
Consider a fixed list l of all words  * . The
  correspondence is formed as follows: For
  every infinite binary sequence S, corresponds
  the language:
       LS    i  | where si  1
                l
                                          QED


36
The Language ___ Is Undecidable
             ATM

So far we proved the existence of a language
   which is not Turing recognizable. Now we
   continue our quest to prove:
Theorem
The language
                                             
            ATM  M , w  M is a TM accepting w
     is undecidable.
37
The Language ___ Is Undecidable
             ATM

Before we start the proof let us consider two
   ancient questions:
Question1:
Can god create a boulder so heavy that god
  cannot lift?



38
The Language ___ Is Undecidable
             ATM

Question2:
In the small town of L.J. there is a single barber:
Over the barber’s chair there is a note saying:
  “I will shave you on one condition: Thaw
  shall never shave thyself!!!”
Who Shaves the Barber?


39
Proof
Assume, by way of contradiction, that ATM is
   decidable and let H be a TM deciding ATM .
That is H  M , w   accept if M accepts w
                      
                       reject if M rejects w

Define now another TM new D that uses H as a
   subroutine as follows:


40
Proof
Define now another TM new D that uses H as a
   subroutine as follows:
D=“On input M where N is a TM:
      1. Run H on input M , M     .
      2. Output the opposite of H’s output
     namely: If H accepts reject, otherwise
     accept.“
41
Proof
Note: What we do here is taking advantage of
  the two facts:
Fact1: TM M should be able to compute with
   any string as input.
Fact2: The encoding of M, M , is a string.



42
Proof
Running a machine on its encoding is analogous
  to using a compiler for the computer
  language Pascal to, that is written in Pascal,
  to compile itself.
As we recall from the two questions self-
   reference is sometimes means trouble (god
   forbid…)

43
Proof
What we got now is:
             accept if M rejects M
            
             
        D M 
              reject if M accepts M
             

Consider now the result of running D with
  input D . What we get is:
             accept if D rejects D
           
        D D 
              reject if D accepts D
             
44
Proof
              accept if D rejects D
            
         D D 
               reject if D accepts D
              
So if D accepts, it rejects wand if it rejects it
   accepts. Double Trouble.
And it all caused by our assumption that TM H
  exists!!!



45
Proof Review
1. Define ATM   M , w  M is a TM accepting w .
2. Assume that ATM id decidable and let H be a
   TM deciding it.
3. Use H to build TM D that gets a TM encoding
    M and behaves exactly opposite to H’s
   behavior,              accept if M rejects M
                 D M   
                          
   namely:                 reject if M accepts M
                             
46
Proof Review
4. Run TM D on its encoding D and conclude:
               accept if D rejects D
             
          D D 
                reject if D accepts D
               
     Contradiction.




47
So Where is the Diagonalization?
The following table describes the behavior of
   each machine on its encoding:
           M1       M2       M3       M4      
     M1   accept            accept            
     M2   accept   accept   accept   accept   
     M3                                       
     M4   accept   accept                     
                           

48
So Where is the Diagonalization?
This table describes the behavior of TM H.
   Note: TM H rejects where M i loops.
           M1       M2       M3       M4      
     M1   accept   reject   accept   reject   
     M2   accept   accept   accept   accept   
     M3   reject   reject   reject   reject   
     M4   accept   accept   reject   reject   


49
Proof Review
Now TM D is added to the table…
           M1       M2       M3       M4           D
     M1   accept   reject   accept   reject    accept
     M2   accept   accept   accept   accept    accept
     M3   reject   reject   reject   reject      reject
     M4   accept   accept   reject   reject    accept
                                         
     D    reject   reject   accept   accept       ???
                                                      
50

								
To top