Docstoc

Introduction to Neural Networks

Document Sample
Introduction to Neural Networks Powered By Docstoc
					INTRODUCTION TO
NEURAL NETWORKS

         By
Dr. M. Tahir Khaleeq


        Total Slides 25   1
The Biological Neuron




                        2
Description of Brain

• The neuron is the fundamental functional unit of the
  brain

• The dentrites are fibers that branch out from the cell
  body into a network around the cell

• The axons are longer and connect the neuron to the
  dentrites of other neurons

• The connection between axon and dentrite is called
  synapse
                                                           3
Function of Brain

• Signals are chemically transmitted from a neuron along
  the axon, reaching the synapses connected to it

• The synapses increase or decrease electric potential to
  the neuron

• When the potential is higher than a given threshold, the
  neuron “fires”, i.e. it sends an action potential down to
  its axons


                                                         4
     Artificial Neural Networks
                (ANN)
• An Artificial Neural Network is a biological inspired
  computational model consisting of processing elements
  (called neurons) and connections between them with
  the coefficients (weights) bounds to the connections,
  which constitute the neuronal structures.

• Neural Networks are called connectionist models
  because of the main role of connections in them.

• The neural networks have similarities to the human
  brain but they are not model of it.
                                                       5
Similarities

• The Artificial Neuron receives inputs that are analogous
  to the electrochemical impulses that the dendrites of
  biological neurons receive from other neurons.

• The output of the artificial neuron corresponds to signals
  sent out from a biological neuron over its axon.

• These artificial signals can be charged similarly to the
  change occurring at the synapses.
                                                         6
     ANN Computational Model
• ANN is a computational model defined by four
  parameters:
   1. Type of Neurons
       – Also called nodes, as a neural network resembles
         a graph.
   2. Connectionist Architecture
       – The organization of the connections between
         neurons
   3. Learning Algorithm
   4. Recall Algorithm                                 7
8
The Neural Model




                   9
INPUTS
– Each input corresponds to a single attribute.
– Neural computing can process only numbers so the
  numeric value of an attribute is used as the input to the
  network.
– If a problem involves qualitative attributes or pictures,
  they must be preprocessed to numerical equivalence
  before they can be treated by ANN.
  EX:
   – Pixel values of characters and groups.
   – Digital images and voice patterns
   – Digital signals from monitoring and control systems.
                                                        10
WEIGHT
– Weight is an key element in an ANN.
– Weights express the relative strength (or mathematical
  value) of each input to a processing element.
– Weights are repeated adjusted, called learning.

INPUT FUNCTION (Summation Function)
– It calculates the aggregated net input signal to the neuron.
– The formula for n inputs is
                    U =  xi wi      i = 1 to n
   Where x and w are the inputs and corresponding weights.
                                                    11
For several neurons:
        n
Uj =  xi wij             j = 1…m
                          n  inputs m  neurons
        i=1



EX:

              w11
x1
                     n1        u1
                                       u1 = x1w11 + x2w21
  w21          w12
                                       u2 = x1w12 + x2w22
 x2                  n2        u2
              w22
                                       u3 = x2w23
         w23
                     n3        u3
                                                       12
ACTIVATION FUNCTION (Transfer Function)
– It calculates the actuation level of the neuron
– Based on the level, the neuron may or may not produce
   an output.
– The relationship between the internal activation level
   and the output may be linear or non-linear.
– Such relationships are expressed by activation function,
   s, as
                     a = s(u)
– There are several types of activation functions.
– The selection of the activation function determines the
   network’s operations.
                                                       13
– The most used activation functions are
  1. The hard-limited threshold function.
  2. The linear threshold function.
  3. The sigmoide function (s-function).
  4. Gaussian function (bell shape function).
– A transformation can occur at the output of each
  processing element, or it can be performed at the final
  output of the network.

OUTPUT FUNCTION
– It calculates the output signal value emitted through the
  output of the neuron O = g(a)
– The output signal is usually assumed to be activation
  level of the neuron, that is, O = a                   14
         1 x  t             1 x0                  1
g ( x)            g ( x)              g ( x) 
         0 x  t             1 x  0            1  e x

                                                              15
Example-1
– Inputs    x1 = 3,       x2 = 1,        x3 = 2.
– Weights w1 = 0.2, w2 = 0.4,            w3 = 0.1.
– Calculation of input function:
              3
      u = = 1 xi wi
          i

          = x1w1 + x2w2 + x3w3
          = 3(0.2) + 1(0.4) + 2(0.1) = 1.2
– Calculation of activation function (s).
                                       1          1
  Apply sigmoid function S( u)           u       
                                    1 e       1  e 1.2
– Output:
                   O = g(0.77) = 1.
                                                           16
Example-2
  ANN with four input nodes, two intermediate nodes
  and one output node. Use he hard-limited threshold
  function.




                                                       17
                 Hard-limited threshold function
             0


             1




                                                   U
                 0
                     1   2    3    4   5


• U(n5) = 1(-5) + 0(3) + 1(2) + 0(4) = -3<3 = 0
• U(n6) = 1(6) + 0(1) + 1(-2) + 0(5) = +3=3 = 1
• U(n7) = 0(-1) + 1(2) = +2<3 = 0
                                                       18
     Connectionist Architecture
• Type of connections between neurons in a neural
  network defines its topology.
• Neurons in a neural network can be fully connected.
   – Every neuron is connected to every other one.
• Neurons can be partially connected,
   – Only connections between neurons in different layers
     are allowed, or
   – in general not all the possible connections between
     all the neurons of the neural network are present. 19
• The connectionist architectures can be distinguished
  according to the number of input and output sets of
  neurons and the layers of neurons used. Following are
  two major connectionist architectures:
   1. Auto-associative
      – Input neurons are output neurons too
      – Hopfield network is an auto-associative network.
   2. Hetro-associative
      – Separate sets of input neurons and output neurons
      – Ex: Perceptron, Multilayer Perceptron.
                                                     20
• The connectionist architectures can also be
  distinguished according to the absence or presence of
  feedback connections. Two types of architectures are
   1. Feed forward Architecture
   – No connections back from the output to the input
     neurons.
   – The network does not keep a memory of its previous
     output values and the activation states of its neurons.
   – Ex: Perceptron-like networks.
   2. Feedback Architecture
   – There are connections back from the output to the
     input neurons
   – Such network keeps a memory of its previous states 21
– The next state depends on the input signals and on
  the previous states.
– Ex: Hopfield network.

                  feedback




                                             Output


    Inputs




                                                  22
                  LEARNING
• Ann ANN learns from its experiences
• The usual process of learning involves three tasks:
   1.Compute outputs.
   2.Compare outputs with desired targets.
   3.Adjust the weights and repeat the process.
• More than a hundred learning (training) algorithms are
  available for various situations and configurations.
• Types of learning algorithms.
   1. Supervised Learning
   2. Unsupervised Learning
   3. Reinforcement Learning                           23
               Compute
                Output




          No   Is Desired
Adjust           Output
Weights        Achieved?


                    Yes

                 Stop


Learning Process of an ANN   24
END


      25

				
DOCUMENT INFO
Shared By:
Stats:
views:33
posted:10/5/2010
language:English
pages:25