Handwritten Characters Based on Recognition by MissPowerPoint

VIEWS: 236 PAGES: 39

• pg 1
```									Handwritten Characters Recognition Based on an

HMM Model
8/12/2008 1

Work Based-on Articles




“A Study of Hidden Markov Models for Offline Recognition of Handwritten Characters” . B. Gosselin & A. Paggiaro “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”. Lawrence R. Rabiner
2

8/12/2008

Project Goal
Application of the Principles Described in Article [1] ,Using Theory From Article [2].
8/12/2008 3

Elements of an HMM
N , the Number of States in the Model. Denoted as S  s1 , s2 ,..., s N . 2. M , the Number of Distinct Observation Symbols Per State , Denoted as V  v1 , v2 ,...,vM . 3. The State Transition Probability Distribution, Denoted as A  aij , Where aij  Pr(qt  s j | qt 1  si ) .
1.









 

8/12/2008

4

Elements of an HMM (Cont.)
4. The Observation Symbol Probability Distribution in State j , Denoted as B  {b j ( k )} where , b j ( k )  Pr(Ot  vk | qt  s j ). 5. The Initial State Distribution , Denoted as Π  { i } Where ,  i  Pr(q1  si ) . 6. The Model Notation is :

  { A, B, }

8/12/2008

5

3 Basic Problems of HMM

Problem 1 ( Evaluation ) How Do We Efficiently Compute

Pr(O |  )?

Solution to This , Allows Us to Choose the Model Which Best Matches the Observations Sequence , O .

8/12/2008

6

3 Basic Problems of HMM (Cont.)
Problem 2 ( Decoding ) Given Sequence , O , and a Model ,  , How Do We Choose Optimally the State Sequence , Q ,Best Matches the Observations , O ? Solution to This , Enables Us to Learn About the Physical Meaning of the Hidden Part of the Model ( e.g the States ).

8/12/2008

7

3 Basic Problems of HMM (Cont.)
Problem 3 ( Learning ) How Do We Adjust the Model Parameters A, B,  in Order to Maximize Pr(O |  ) Per Each Class of Characters ? Solution to This , Enables Us to Optimize The Model Parameters , In Order to Have Best Description of How a Given Observation Sequence Comes About .

8/12/2008

8

Creating My Database
1. Creating Digital Image of Handwritten

Characters. 2. Isolation of the Characters. 3. Binarization ( Using Local Threshold ).

3
8/12/2008

2

1
9

My Database Specifications
• • • •

Samples From 8 Different People. 918 Characters ( ~ 8 x 5 x 27 ). Character’s Size - 0.7 x 0.6 [Cm] , ( 0.24 x 0.27 [Inch] ). Scanner – 300 DPI , 256 Gray-Level .
8/12/2008 10

Applying HMM Procedure Top Level
Binary Images
Pre Process Feature Extraction

Designing Discrete
HMMs

8/12/2008

11

Pre - Processing
Goal –  Provide Skeleton Image of Each Character.
Binary Images

Invert Colors

Create Skeleton Algorithm

8/12/2008

12

Examples Of Skeletons

8/12/2008

Original

Skeleton

13

Feature Extraction


Reduce Amount of Data Held in Characters.  Provide Attractive Representation (Using Oriented Search Principle Method).

95 Pairs (X,Y)
8/12/2008

14 Pairs (X,Y)
14

Oriented Search Principle
Goals –
 “Chipper” Characters Representation. 
1. 2. 3.

Trying to Retrieve Writing Dynamics.

Algorithm Steps –
Finding First Pixel ( TBLR Scan ). Specifying the Next Likely Pixel ( Iteratively ). Conclusion of Line Segment. ( By Distortion TH or Number of Pixels ). 4. Normalizing Ordinates ( According to the Initial Width & Height of Character ).
8/12/2008 15

Oriented Search Principle (Cont.)
Illustration –

8/12/2008

16

Feature Extraction - Example
0
60

Observation Sequence
60
X1

5

15 50 35 30 30 40

O

Y1 X2 Y2

=

55 40 25 25 30 35 45 15 15 35 30 30 40 60 40 5 25 30 35 45 45
17

8/12/2008

Example of Extremities Ordinates Distribution

8/12/2008

18

Feature Extraction - Summarize
  

  

Preserve Shape of Characters. Reduce Amount of Data. Retrieve Dynamic Level of Writing.

Over Sensitivity to First Pixel. Significant Variance of Segments Number. Significant Variance of Segments Order.

Requires HMM Design !!
8/12/2008 19

Designing Discrete HMM ( Per Each Class )
Observations Sequences Create ACM Initialize HMM Structure Training HMM

8/12/2008

20

Designing Discrete HMM ( Cont. )

Average Character Model
A Representative Model For Each Character Defined By All Obs.

Goals
  

Defines the Number of States in the HMM Structure. Initializes the State Transitions Distributions. Gives Physical Meaning to the States of the HMM ( Each State Correspond to Segment ).
8/12/2008 21

Designing Discrete HMM ( Cont. )

Average Character Model (Cont.)
Creation Procedure :
1. 2. 3. 4.

5.

Definition of Segments Number, TH. Applying VQ Process ( Local,Mean+Std ). Association of Segments to ACM Segments.( Using Distance Measure ). Updating ACM ( According to Calculation of Mean Value of All Grouped Segments ). Goto Step 3 ( Unless TH Achieved ).
22

8/12/2008

Designing Discrete HMM ( Cont. )

Average Character Model (Cont.)
Distance Measure Definition

: d  min( d1 , d 2 )

1 2 1 2 1 2 1 2 d1  ( x0  x0 ) 2  ( y0  y0 ) 2  ( x1  x1 ) 2  ( y1  y1 ) 2 1 2 1 2 1 2 1 2 d 2  ( x1  x0 ) 2  ( y1  y0 ) 2  ( x0  x1 ) 2  ( y0  y1 ) 2

d1
8/12/2008

d2
23

Designing Discrete HMM ( Cont. )

Initialize HMM Structure
N = Segments On ACM.  Si  ith ACM Segment. (Random Selection) Example:

av av {O1av , O2 ,...,ON }

{S1 , S 2 ,.......,S N }
8/12/2008 24

Designing Discrete HMM ( Cont. )

Segments Alignment Process
Goal Matching Each Segment of Each Character to the Ones of the ACM.

Training Char.
8/12/2008

ACM
25

Designing Discrete HMM ( Cont. )

Alignment Process Results
Each Observations Sequence’s Segment Is Indexed According to ACM Closest Segment. Example : {O1} AP {S , S ,...,S } 
1 1 N

{O }  {S 4 , S3 ,...,S1}
2 AP

{O } {S 2 , S1 ,...,S 5 }
K AP
8/12/2008

O [ 4KxT ]

S [ KxN ]

26

Designing Discrete HMM ( Cont. )

Initialize HMM Parameters
According to Matrix S :
Number of S i ' s in First Column of S  Initial {  }   i  K Transition s from S i to S j  Initial { A }  aij  Transition s from S i to Any State


Initial { B } 

b j (k ) 

Observations VK in S j Total Observations in S j
27

8/12/2008

Designing Discrete HMM ( Cont. )

Training HMM Procedure
Training Data {O}

Q  {q1 , q2 ,...,qT } Max{P(Q | O,  ) }

Calculate

( Viterbi Path )
8/12/2008

  { A, B, }
28

Designing Discrete HMM ( Cont. )

Training The HMM ( Cont.)

Dist ( ,  )  Th

?

 

Yes
Save   
8/12/2008

No

29

Classification System
HMM ‫א‬

P(O |  )
MLP

‫א‬

Skeleton Char.

HMM Feature Extraction ‫ ב‬P(O |  )

Image HMM ‫ת‬
8/12/2008

‫ת‬
30

P(O |  )

Improve Classification Performance
Classification Decision Calculation Using the Forward-Backward Procedure :

P (O |  )    T (i )
i 1

N

 t (i )  P (O1 ,.....,Ot , qt  S i |  )
N2 T Calculations Vs.
8/12/2008

2T NT Calculations !!
31

Improve Classification ( Cont. )
1. Adding the Probability That an Observations
Sequence ends on a given State Si :

P(O |  )   ( T (i )   f (i ))
i 1

N

 f (i )  P(qT  Si )
8/12/2008 32

Improve Classification ( Cont. )
2. Adding the Probability That the Observations
Sequence Length is T :

N  P(O |  )   ( T (i )   f (i ))  n(T )  i 1  n( x)  P(T  x)

8/12/2008

33

Improve Classification ( Cont. )
3. Adding “Punishment” Probability to Each
State, Si ,of the Model ,That Doesn’t Take Place in a Sequence Of Observations : ( Improve Discrimination [ ‫. ) ] ר,ב‬  N  P(O |  )   ( T (i )   f (i ))  n(T )   (1  c(i )) *  i 1  i , S{ qt } c(i )  P( Si  {q j }) , 1  i  N , 1  j  T
{qt*}

 Viterby Optimal State Sequence.
34

8/12/2008

Improve Classification ( Cont. )
4. Adding the Probability Distribution of the
“Discrete” Aspect Ratio ( Width / Height ). ( Avoid Discrimination Problem Due to the Normalization of Character Dimensions ).

N  P(O |  )   ( T (i )   f (i ))  n(T )   (1  c(i ))  r ( R)  i 1  i , S{qt* } r ( X )  P(r  X ) , X  0.2 : 0.1 : 2
8/12/2008 35

Classification ( DB Properties )
 




Performed on NIST3 Database. 1579 of Each Class English Unconstrained Uppercase Handwritten Characters. 1324 – Training Set, 235 – Testing Set. Total Number of Characters : 41,054. ( 1579 X 26 ).
8/12/2008 36

Classification ( Total Results )
Improve Number

None

1

2

3
83.2

4
84.4

Recog. Rate (%)

78.6 79.6 80.3

8/12/2008

37

Classification ( Results Per Char. )

8/12/2008

38

My Goals
Implementation of those Principles (on MATLAB). Using My “Small” Database. Having Same Recognition Rate.
8/12/2008 39

```
To top