Neural networks Artificial Intelligence Methods

Document Sample
Neural networks Artificial Intelligence Methods Powered By Docstoc
					                  Neural networks

                       Ludwik Liszka

                  La Londe October 8-13, 2001


                                                                1




     Artificial Intelligence Methods



Collection of simulated cognitive processes and not the intelligence
in real meaning,




                                                                2




                                                                       1
     Biological Neuron




                               3




Analytical model of a neuron




                               4




                                   2
      A simple neural network




                                               5




            Network learning
• Supervised learning;   • Unsupervised
                           learning:
Back-Propagation         Self-Organizing Map
  Network (BP)             Netwok (SOM)




                                               6




                                                   3
    Back-Propagation Network
• Preferably not more than 1 hidden layer
• Careful dimensioning of the hidden layer
• Selection of the Epoch
• Decide what is the nature of the output
  (unipolar/bipolar) – that will decide the type
  of transfer function in the output layer
• Dynamic range of the network!!!
                                               7




    Back-Propagation Network
• Learning requires desired output
  (supervised learning)




                                               8




                                                   4
    Applications of the BP network

•   Pattern recognition (classification)
•   Function fitting
•   Filling data gaps
•   Coordinate transformation



                                            9




    Self-Organizing Map Network




                                           10




                                                5
            SOM Network
• Autoassociative network
• Main application: pattern recognition
  (categorization)
• Learning does not require desired output
  (unsupervised learning)



                                             11




      Neural networks – input
          preprocessing


• Transformations
• FFT or Wavelet Transform
• Principal Component Analysis



                                             12




                                                  6
Example: pattern recognition
 Assume a pattern consisting of simple elements:




                                                   13




Example: pattern recognition




                                                   14




                                                        7
   Example: pattern recognition




• Window 3 x 3 pixels, recognition with a single BP

                                                  15




   Example: pattern recognition




                   • SOM + BP
                                                  16




                                                       8
       Example: pattern recognition




                      • PCA + SOM + BP
                                                             17




      Pattern recognition with PCA




Projections of analyzing windows in the directions of PC1 and PC2
                                                              18




                                                                    9
A comparison of all 3 techniques




                                               19




 Example of pattern recognition
• Identification of Lower Hybrid Cavities in
   Freja data
J. Waldemark, PhD Thesis, 1995




                                               20




                                                    10
               Data modelling

• Time series prediction
• Multivariate process modelling
• Self-adjusting models




                                                           21




 Prediction of time series with a BP




• Prediction of Wolf numbers one day ahead with a 32-day
  window
                                                           22




                                                                11
         Example: function fitting




                                                            23




         Example: function fitting




Surface fitted with a BP (2PE in, 1PE out + hidden layer)
                                                            24




                                                                 12
Dimensioning the hidden layer




                                25




      Original function




                                26




                                     13
Modelling of a multivariate process




                                  27




     Self-adjusting models




                                  28




                                       14
TETRAD – a causal modelling technique

• Search for causal relations in multivariate
  data
• The method is based on covariance- or
  correlation matrix for the data
• Possible applications: crosstalk between
  experiments, source identification, ...


                                                29




                                                     15