# Midterm â€“ CAP 5638 Pattern Recognition

Document Sample

```					Midterm – CAP 5638 (Spring 2002)
Name: _________________________                                           Initials: __________
Last Four of SSN: _________________

Midterm – CAP 5638: Pattern Recognition
Department of Computer Science, Florida State University, Spring 2002


Problem 1. (10 points, 2 points each) True/False. For each statement, write ‘T’ at the end if the
statement is true and write ‘F’ if the statement is false.
1) In a two-category case, if p(x|1) and p(x|2) are normal distributions with the same
variance but different means, there exists a linear discriminant function which gives the
same decision for all inputs as the minimum error rate classifier……………... [           ]
2) Given that all p(x|i) are normal distributions, the decision region for each i from the
minimum error rate classifier is always a single connected region. …….….…. [           ]
3) Regardless of the true underlying model, the minimum error rate classifier using the
maximum likelihood estimate gives always the smallest average classification error
within the assumed model …………………………………….……………….. [                                   ]
4) Let D={x1, …., xn} be a training set with n > 2. The mean estimated using (2x1+x2)/3 is
unbiased ………………………………………………………..………….… [                                           ]
5) Assume that there is only one parameter in a probability model, the Bayesian parameter
estimation always gives one unique solution., i.e., a number ….…….. …….... [      ]

Problem 2. (12 points, 3 points each) Short answers.
1) Define and explain the Bayes formula.

2) Describe the minimum error rate classifier.

3) Explain the nearest-neighbor rule for classification.

4) Explain the Bayesian parameter estimation technique.

1
Midterm – CAP 5638 (Spring 2002)                                                                        Initials: __________

Problem 3. (16 points, 4 points each) Suppose that there are two classes, 1 and 2 and a one-
dimensional feature x. The likelihood functions for the two classes are given as follows:

1                                                 1
exp ( x 2)                                      exp ( x  2)
2                                                  2
p( x | 1 )                            /2
;   p( x |  2 )                             /2
.
2                                                2

1) Given P(1 )  1 / 2 and P( 2 )  1 / 2 , calculate the minimum error rate classification decision
and its error for x = -8 and x = 8.

2) Given P(1 )  0.1 and P( 2 )  0.9 , find analytically the decision boundary and decision
regions.

3) Derive a discriminant function in terms of x, P(1 ) and P( 2 ) that gives the minimum
classification error and can be computed efficiently.

4) Given P(1 ) and P( 2 ) , show that the minimum error rate decision for a particular x
depends only on p(x|1) / p(x|2).

2
Midterm – CAP 5638 (Spring 2002)                                                            Initials: __________

Problem 4. (15 points) Derive the Bayes classification error for the following two-category
models with P(i)=1/2.
 1/ 2    if 0  x  2r

p( x |  i )   1      if i  x  i  1 - r

1) (8 points) For 0 < r < 0.5,                              0      otherwise

2) (7 points) p( x |  i )  3 / 2 for i/3  x  (i  2)/3 .

       0      otherwise

3
Midterm – CAP 5638 (Spring 2002)                                                  Initials: __________

Problem 5. (15 points) Let x be a d-dimensional binary vector with a multivariate Bernoulli
distribution
d
p( x|  )    
i 1
xi
(1   ) 1 xi

where  is an unknown parameter. Given training samples D1={x1, x2, …, xn} for 1 and D2={y1,
y2, …, yn} for 2, answer the following questions.
1) (3 points) Write the log-likelihood functions p(D1|) and p(D2|2).

2) (5 points) Derive the maximum likelihood estimate for  and 2. You need to show all
the steps.

3) (2 points) Give the maximum likelihood estimate for P(1) and P(2). No derivation is
needed.

4) (5 points) Using the above results, give a linear discriminant function in terms of xi that
gives the minimum classification error.

4
Midterm – CAP 5638 (Spring 2002)                                              Initials: __________

Problem 6. (15 points) Given a training set D={1, 2, 2, 3, 5} and a window function
1
 (u )         exp{u 2 / 2}
2                   , answer the following questions.
1) (6 points) Given the Parzen windows estimate of p(x).

2) (3 points) Let the window width hn be very close to 0, sketch estimated p(x) in the
following diagram.

3) (3 points) Let the window width hn be very large, sketch estimated p(x) in the following
diagram.

4) (3 points) Based on your results, explain why both limiting values of hn are not desirable.

5
Midterm – CAP 5638 (Spring 2002)                                         Initials: __________

Problem 7. (12 points) Four categories are represented by their sample means
m1=[-1 –1], m2=[1 1], m3=[-1 1] and m4=[1 –1].
1) (7 points) Mark and label the means, and then draw the decision boundary and decision
regions resulting from the nearest neighbor rule in the following diagram.

2) (5 points) Give p(x|i) and P(i) whose minimum error rate classification leads to the
same decision regions. You need to specify all the parameters. (Hint: Think about normal
distributions).

6
Midterm – CAP 5638 (Spring 2002)                                         Initials: __________

Problem 8 (15 points, 5 points each) Given a training set D={x1, …, xn}, derive the maximum
likelihood estimate for the following models.

 exp{ x} if x  0
p( x |  )  
1)                 0         othersize


2 x exp{ x 2 } if x  0
p( x |  )  
0
                 othersize
2)

1              ( x  10) 2
p( x |  )           exp{                 }
3)                  2               2 2

7

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 119 posted: 6/18/2010 language: English pages: 7
How are you planning on using Docstoc?