PowerPoint Presentation by HC12091806498

VIEWS: 0 PAGES: 70

									                    Minimal bounds on the Mean Square Error: A Tutorial




                                    Alexandre Renaux
                            Washington University in St. Louis




Friday 23rd February 2007                  1/70                    Alex
                                                       Outline

       Minimal bounds on the Mean Square Error
      1       Framework and motivations

      2       Minimal bounds on the MSE: unification

      3       Minimal bounds on the MSE: application

      4       Conclusion and perspectives




Friday 23rd February 2007           2/70                         Alex
                                                       Outline



      1       Framework and motivations

      2       Minimal bounds on the MSE: unification

      3       Minimal bounds on the MSE: application

      4       Conclusion and perspectives




Friday 23rd February 2007           3/70                         Alex
                                      1     Framework and motivations


                              Statistical signal processing

                                             Extract informations
                                                 (estimation)


                            Applications:    Radar/Sonar
                                             Digital communications
                                             Medical imaging
                                             Astrophysic
                                             …


Friday 23rd February 2007                        4/70                   Alex
                              1   Framework and motivations

           Statistical framework



                 Parameters
                                                  Observations
                    space
                                                     space




Friday 23rd February 2007             5/70                       Alex
                                            1   Framework and motivations

                         Performances

                            r.v.          ’’Distance’’ between      and
                                                Mean Square Error



                                   Bias
 Estimates distibution




                                                 Variance




Friday 23rd February 2007                             6/70                  Alex
                                            1    Framework and motivations

                              Performances: Cramér-Rao inequality


                    For               unbiased




                                                             with
     Estimates distribution




                                         Cramér-Rao                 Fisher Information
                                                                          Matrix

                                                                     If equality, then
                                                                    efficient estimator

Friday 23rd February 2007                             7/70                                Alex
                                  1   Framework and motivations
             Context

           Maximum Likelihood estimators




                                Direction of Arrivals estimation
                                Frequency estimation

                            The parameters support is finite




Friday 23rd February 2007                  8/70                    Alex
                                                1   Framework and motivations

                                                                               Rife and Boorstyn 1974
     MSE behavior of ML estimator: 3 areas                                     Van Trees 1968


                                        Non-Information
               Mean Square Error (dB)




                                                                 SNRT   Signal to Noise Ratio (dB)


Friday 23rd February 2007                                 9/70                                       Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                     SNR = 10 dB
                                                                     Run 1




                                                                   MSE
                                                                                     SNR


                              Normalized frequency
Friday 23rd February 2007                    10/70                                 Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                     SNR = 10 dB
                                                                     Run 2




                                                                   MSE
                                                                                     SNR


                              Normalized frequency
Friday 23rd February 2007                    11/70                                 Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                     SNR = 10 dB
                                                                     Run 20




                                                                   MSE
                                                                                     SNR


                              Normalized frequency
Friday 23rd February 2007                    12/70                                 Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -6 dB
                                                                         Run 1




                                                                   MSE
                                                                                       SNR


                              Normalized frequency
Friday 23rd February 2007                    13/70                                     Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -6 dB
                                                                         Run 2




                                                                   MSE
                                                                                       SNR


                              Normalized frequency
Friday 23rd February 2007                    14/70                                     Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -6 dB
                                                                         Run 7

                                        Outlier




                                                                   MSE
                                                                                       SNR


                              Normalized frequency
Friday 23rd February 2007                    15/70                                     Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -20 dB
                            Outlier                                      Run 1




                                                                   MSE
                                                                                        SNR


                               Normalized frequency
Friday 23rd February 2007                     16/70                                     Alex
                                     1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -20 dB
                        Outlier                                          Run 2




                                                                   MSE
                                                                                        SNR


                                  Normalized frequency
Friday 23rd February 2007                        17/70                                  Alex
                                  1    Framework and motivations
                  Single frequency estimation (100 observations)

                                                                         SNR = -20 dB
                                                                         Run 20
                                                 Outlier




                                                                   MSE
                                                                                        SNR


                              Normalized frequency
Friday 23rd February 2007                    18/70                                      Alex
                                           1    Framework and motivations


                                                              - Asymptotic MSE
                                                              - Asymptotic efficiency
                              Non-Information                 - Threshold prediction
     Mean Square Error (dB)




                                                              - Global MSE
                                                              - Ultimate performances




                                                 SNRT      Signal to Noise Ratio (dB)



Friday 23rd February 2007                          19/70                                Alex
                                                       Outline



      1       Framework and motivations

      2       Minimal bounds on the MSE: unification

      3       Minimal bounds on the MSE: application

      4       Conclusion and prospect




Friday 23rd February 2007           20/70                        Alex
                           2 Minimal bounds on the MSE: unification


                 Mean Square Error (dB)                               Insuffisancy of the Cramér-
                                                                       Rao bound
                                          Non-Information
                                                                           - Optimistic
                                                                           - Bias
                                                                           - Threshold

                                                                                 Other minimal
                                                                                    Bounds
                                                                                  (tightest)



                                                               SNRT    Signal to Noise Ratio (dB)



Friday 23rd February 2007                                   21/70                                   Alex
                   2 Minimal bounds on the MSE: unification

                 Two categories

                   Deterministic             Random
                    parameters             parameters



         Determininstic Bounds              Bayesian Bounds


      Bound the local MSE                  Bound the global MSE




Friday 23rd February 2007          22/70                          Alex
                   2 Minimal bounds on the MSE: unification

                 Two categories

                   Deterministic             Random
                    parameters             parameters



         Determininstic Bounds              Bayesian Bounds


      Bound the local MSE                  Bound the global MSE




Friday 23rd February 2007          23/70                          Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification               Glave IEEE IT 1973




  Barankin Approach

                   In a class of unbiased estimator      , we want to find the
                   particular estimator for which the variance is minimal
                   at the true value of the parameter


                                Constrained optimization problem

                               Class      of unbiased estimator ????




Friday 23rd February 2007                   24/70                                Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                              Barankin (1949)




Friday 23rd February 2007            25/70               Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




           Barankin




                            Needs the resolution of an integral equation
                            Sometimes,      doesn’t exist



Friday 23rd February 2007                    26/70                         Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                                Cramér-Rao




Friday 23rd February 2007           27/70                Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




         Cramer             Rao   Fisher           Frechet   Darmois




Friday 23rd February 2007                  28/70                       Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                              Bhattacharyya
                                 (1946)




Friday 23rd February 2007           29/70                Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                             Bhattacharyya      Barankin




Friday 23rd February 2007            30/70                 Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification



                                               ?

          Bhattacharyya     Guttman           Fraser




Friday 23rd February 2007             31/70              Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                            McAulay-Seidman     (Barankin)
                                (1969)


                                                             Test points

Friday 23rd February 2007               32/70                              Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                            McAulay-Seidman     Barankin



                                                    Test points

Friday 23rd February 2007              33/70                      Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




           How to choose test points ?




Friday 23rd February 2007           34/70                Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                            Chapman-Robbins (1951)
                                 1 test point




Friday 23rd February 2007              35/70             Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




          Chapman           Robbins   Hammersley           Kiefer




Friday 23rd February 2007                          36/70            Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                               Abel (1993)




                                                Test points

Friday 23rd February 2007           37/70                     Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




Friday 23rd February 2007           38/70                Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds unification




                       Quinlan-Chaumette-Larzabal (2006)




                                                           Test points

Friday 23rd February 2007                  39/70                         Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds
                                            f0=0, K=32 observtions


                                              Don’t take into accout the
                                              support of the parameter




Friday 23rd February 2007           40/70                              Alex
                   2 Minimal bounds on the MSE: unification

             Deterministic bounds

    Already used in Signal Processing

          CRB for wide range of topics

          ChRB and Barankin (McAulay-Seidman version)

                            Time delay estimation
                            DOA estimation
                            Digital communications (synchronization parameters)

           Abel bound
                            Digital communications (synchronization parameters)


Friday 23rd February 2007                    41/70                                Alex
                   2 Minimal bounds on the MSE: unification

                 Two categories

                   Deterministic             Random
                    parameters             parameters



         Determininstic Bounds              Bayesian Bounds


      Bound the local MSE                  Bound the global MSE




Friday 23rd February 2007          42/70                          Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification



               Best Bayesian bound: MSE of the conditional mean estimator
                                     (MMSEE)




    is the solution of




Friday 23rd February 2007                43/70                              Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




    For your information




Friday 23rd February 2007           44/70                Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




               Best Bayesian                   Minimal
                  bound                        bound


Friday 23rd February 2007           45/70                Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




                            Constrained optimization problem

                            Degres of freedom

Friday 23rd February 2007                       46/70          Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification
                                   s
                                  1




                                                                     h
                                               Best Bayesian bound


Friday 23rd February 2007              47/70                         Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification
                                               s
                                              1




                                                                         h
                            Bayesian Cramér-Rao bound (Van Trees 1968)


Friday 23rd February 2007                          48/70                 Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




     Van Trees




Friday 23rd February 2007           49/70                Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification
                                               s
                                              1
                                                           …

                                                               Test points




                                                                                   h
                            Reuven-Messer bound (1997) (Bayesian Barankin bound)
                            Bobrovsky-Zakaï bound (1976) (1 test point)

Friday 23rd February 2007                          50/70                           Alex
                    2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




           Reuven           Messer   Bobrovsky           Zakai




Friday 23rd February 2007                        51/70           Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification


                                        1




                                                                    h
                    Bayesian Bhattacharyya bound (Van Trees 1968)


Friday 23rd February 2007                   52/70                   Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification




     Van Trees




Friday 23rd February 2007           53/70                Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds unification
                                        s
                                        1




                                                           h
                            Weiss-Weinstein bound (1985)


Friday 23rd February 2007                   54/70          Alex
                   2 Minimal bounds on the MSE: unification

               Bayesian bounds unification




       Weiss                Weinstein




Friday 23rd February 2007               55/70            Alex
                   2 Minimal bounds on the MSE: unification

             Relationship between deterministic and Bayesian bounds

         Deterministic bounds                     Bayesian bounds


                            Constrained optimization problem




Friday 23rd February 2007                 56/70                       Alex
                   2 Minimal bounds on the MSE: unification

             Relationship between deterministic and Bayesian bounds



                    Deterministic bounds               Bayesian bounds

                             Cramér-Rao              Bayesian Cramér-Rao

                            Bhattacharyya           Bayesian Bhattacharyya

                     Chapman-Robbins                    Bobrovsky-Zakaï

                     McAulay-Seidman                     Reuven-Messer

                                Abel                          ???
                                ???                      Weiss-Weinstein


Friday 23rd February 2007                   57/70                            Alex
                   2 Minimal bounds on the MSE: unification

             Relationship between deterministic and Bayesian bounds



                    Deterministic bounds               Bayesian bounds

                             Cramér-Rao              Bayesian Cramér-Rao

                            Bhattacharyya           Bayesian Bhattacharyya

                     Chapman-Robbins                    Bobrovsky-Zakaï

                     McAulay-Seidman                     Reuven-Messer

                                Abel                      Bayesian Abel

                                ???                      Weiss-Weinstein


Friday 23rd February 2007                   58/70                            Alex
                   2 Minimal bounds on the MSE: unification

             Relationship between deterministic and Bayesian bounds



                    Deterministic bounds               Bayesian bounds

                             Cramér-Rao              Bayesian Cramér-Rao

                            Bhattacharyya           Bayesian Bhattacharyya

                     Chapman-Robbins                    Bobrovsky-Zakaï

                     McAulay-Seidman                     Reuven-Messer

                                Abel                      Bayesian Abel

          Deterministic Weiss-Weinstein                  Weiss-Weinstein


Friday 23rd February 2007                   59/70                            Alex
                   2 Minimal bounds on the MSE: unification

             Bayesian bounds
    Already used in Signal Processing
          BCRB and Bayesian Barankin bound ???

          Bobrovsky-Zakai and Bayesian Abel bounds
                            Digital communications (synchronization parameters)

          Weiss-Weinstein bounds
                             Spectral Analysis
                             Underwater acoustic
                             Array Processing
                             Digital communications
                             Physics (gravitational waves)
Friday 23rd February 2007                    60/70                            Alex
                                                       Outline



      1       Framework and motivations

      2       Minimal bounds on the MSE: unification

      3       Minimal bounds on the MSE: application

      4       Conclusion and perspectives




Friday 23rd February 2007           61/70                        Alex
                      3 Minimal bounds on the MSE: application

             Synchronization problem (spectral analysis)



                  Observations (complex)

                  Pilot symbols (known)

                                       Additive noise, Gaussian, circular, iid

                   Parameter of interest


                            deterministic or random



Friday 23rd February 2007                       62/70                            Alex
                      3 Minimal bounds on the MSE: application

             Synchronization problem (spectral analysis)

                                                  QPSK Modulation
                                                  20 observations



                                                      Maximum
                                                      Likelihood

                                                      Local MSE



Friday 23rd February 2007            63/70                         Alex
                      3 Minimal bounds on the MSE: application

             Synchronization problem (spectral analysis)

         Bayesian bounds for deterministic parameters estimators




                            The support of the parameter is taken into
                            account



Friday 23rd February 2007                64/70                           Alex
                      3 Minimal bounds on the MSE: application

             Synchronization problem (spectral analysis)

                                                  QPSK Modulation
                                                  20 observations



                                                      Maximum
                                                      Likelihood

                                                      Global MSE



Friday 23rd February 2007            65/70                         Alex
                                                        Outline



      1       Framework and motivations

      2       Minimal bounds on the MSE: unification

      3       Minimal bounds on the MSE : application

      4       Conclusion and perspectives




Friday 23rd February 2007           66/70                         Alex
                               4    Conclusion and perspectives

        Contributions

           Deterministic bounds unification


           Bayesian bounds unification


           Bayesian Abel bound and deterministic Weiss-Weinstein bound


           Closed-form expressions of the minimal bounds in a
           synchronization framework (+ Gaussian observation model with
           parameterized mean)



Friday 23rd February 2007                67/70                            Alex
                                4   Conclusion and perspectives
                    BTW
       -Understanding the CRB


       - CRB for Singular FIM


       - Ziv-Zakai Family (Bayesian)


       - Minimal bounds for discret time filtering (Bayesian)


       - Some global class of Cramér-Rao bound (Bayesian)


       - Hybrid bounds (mixing non-random and random parameters)
Friday 23rd February 2007              68/70                       Alex
                               4    Conclusion and perspectives

        Perspectives

           Estimators set for Bayesian bounds


           Interpretation in terms of bias of the Weiss-Weinstein bound


           Closed form expressions for the multiple parameters case




Friday 23rd February 2007              69/70                              Alex
               Minimal bounds on the Mean Square Error




                            Thank you




Friday 23rd February 2007       70/70              Alex

								
To top