# Designing Programs that Check Their Work

Document Sample

```					Designing Programs
that Check Their Work
Manuel Blum
Sampath Kannan

by Jeffrey Corbell
Overview
•   Introduction to a Program Checker
•   Other Methods of Determining Correctness
•   Definition of a Program Checker
•   Example of a Checker: Graph Isomorphism
•   Beigel’s Theorem
What is a program checker
• Program that checks the output of a program to
determine if the program is correct or buggy

Formally:
– P and C are programs, I is the input
– For any I run on P, C is run and determines
whether P is correct for I or buggy
Other Methods of Determining
Correctness

• Program verification
– Use a proof to prove a program is correct
– Very difficult to do
– Argued that it doesn't improve confidence in
correctness
• very complex
• may contain errors which would be difficult to detect
Other Methods of Determining
Correctness

• Program testing
– Run program on input that you know the correct
output for
– Compare program output to expected output
– Problems
• No general way to create test data
• No theorems to describe behavior if they do pass tests
Differences Between a Checker and
Testing

• A checker is a program that uses its own
algorithm that allows it to check the output
• Program testing usually only uses a small
amount of predetermined cases for specific
input
Definition of a Bug
• Let π represent a decision or search problem
• x represents an input to π with π(x)
representing the output
• P is a deterministic program that supposedly
solves π

P has a bug if for some instance x of π
P(x) ≠ π(x)
Definition of a Checker
• Let Cπ be the checker, k be the number of
different cases the checker tries, and I be the
group of test inputs
• CπP(I,k) is the output of the checker and
follows these conditions:
1. If P(x) = π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = CORRECT
2. If P(x) ≠ π(x), then with probability ≥ 1- 1/2k
CπP(I,k) = BUGGY
Definition of a Checker

• However, if P has bugs but P(I)=π(I) then
CπP(I,k) may output either CORRECT or BUGGY
Definition of a Checker

• Assumed P halts on all inputs
• Not always the case
• If P(x) exceeds a predetermined bound then
the checker should raise a flag, CπP(I,k) = TIME
Definition of a Checker

• Runtime includes the time it takes to submit
input and receive output from P
• Does not include the time it takes P to run
Definition of a Checker
• If a checker is a program, how can you be sure
the checker is correct?
• You can’t really
• Checker must have the little oh property with
respect to the runtime of P
– Ensures the checker is programmed differently
than the original program
Graph Isomorphism

a
1
f (a) = 1
c
b                   5               4   f (b) = 2
f (c) = 3
f (d) = 4
f (e) = 5
d       e           2       3
Graph Isomorphism Checker
• Let P be a program that solves graph
isomorphism
– Input: two graphs G and H
– Output: YES if G is isomorphic to H; NO otherwise

• CGIP(G, H, k) checks P on input G and H
Graph Isomorphism Checker
• Compute P(G,H)
• If P(G,H)=YES then
– Use P to search for an isomorphism from G to H
– Check if the resulting correspondence is an
isomorphism
– If not, return BUGGY; if yes, return CORRECT
Graph Isomorphism Checker
• If P(G,H)=NO then
– Do k times:
• Toss a fair coin
• If coin = heads then      • If coin = tails then
– Generate a random         – Generate a random
permutation G’ of G         permutation H’ of H
– Compute P(G,G’)           – Compute P(G,H’)
– If P(G,G’)=NO then        – If P(G,H’)=YES then
return BUGGY                return BUGGY

• Return CORRECT
Graph Isomorphism Checker
• CGIP runs in polynomial time
• If P has no bugs and G is isomorphic to H, then
CGIP(G,H,k) creates an isomorphism from G to
H and outputs CORRECT
• If P has no bugs and G is not isomorphic to H,
then CGIP(G,H,k) tosses coins. It discovers
P(G,G’)=YES for all G’ and P(G,H’) for all H’ so
outputs CORRECT
Graph Isomorphism Checker
• If P(G,H) is incorrect then there are two cases:
– If P(G,H)=YES but G is not isomorphic to H, then
CGIP fails to construct an isomorphism and outputs
BUGGY
– If P(G,H)=NO but G is isomorphic to H, the only
way that C will return CORRECT is if P(G,G’)= YES if
the coin is heads and P(G,H’)= NO when it is tails.
But G and H are permuted randomly to produce
G’ and H’. Therefore P correctly distinguishes G’
from H’ only by chance for just 1 of 2k possible
sequences
Beigel’s Theorem
• Let π1 and π2 be two polynomial-time
equivalent decision problems. Then from any
polynomial time checker for π1 it is possible to
construct a polynomial-time checker for π2.
Beigel’s Theorem
• Have a checker Cπ1 for π1 and a program P2 for
π2
• Also have two way polynomial time
transformations f1,2 and f2,1
• This gives us a program for π1
– P1(x) =P2(f1,2(x))
Beigel’s Theorem
• To check P2 on an input y, compute P2(y) then
transform into an input z for π1 using f2,1
• Then use Cπ1 to check z.
• Any call Cπ1 makes to P1 is transformed to a
call to P2
f1,2          P1
P2                               Cπ1
y
f2,1
z
Beigel’s Theorem
• If P2 is correct then P1 will be correct because
P1 is defined in terms of P2
• Thus if P1 is correct on z then P2 is correct on y
• If P2 is wrong on y and P1 is correct on z then
• If P1 is wrong on z then the checker Cπ1 will
catch it
Beigel’s Theorem
• This checker for π2 runs in polynomial time
– Running the checker for π1
– One transformation of f2,1
– Polynomial number of applications of f1,2
Bibliography
• Designing programs that check their work - M. Blum and S.
Kannan
• Social Processes and Proofs of Theorems and Programs - R.A.
De Millo, R.J. Lipton, and A.J. Perlis.
• Introduction to the Theory of Computation – M. Sipser
• www.wikipedia.org

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 12 posted: 12/1/2012 language: English pages: 24