Introduction to Business Analytics - PowerPoint by hjt98841

VIEWS: 132 PAGES: 39

									Introduction to Business Analytics
Chapter 6: Neural Networks for Data Mining

          Matthew J. Liberatore
           Thomas Coghlan

                Fall 2008
        Learning Objectives
 Understand     the concept and different types
  of artificial neural networks (ANN)
 Learn the advantages and limitations of
 Understand how backpropagation neural
  networks learn
 Understand the complete process of using
  neural networks
 Appreciate the wide variety of applications
  of neural networks
Using Neural Networks to Predict Beer
   Flavors with Chemical Analysis
1.   Why is beer flavor important to the profitability
     of Coors?
2.   What is the objective of the neural network
     used at Coors?
3.   Why were the results of the Coors neural
     network initially poor, and what was done to
     improve the results?
4.   What benefits might Coors derive if this project
     is successful?
5.   What modification would you provide to
     improve the results of beer flavor prediction?
    Business Applications of Artificial
        Neural Networks (ANN)
   Many applications across all areas of business
       target customers (CRM)
       bank loan approval
       hiring
       stock purchase
       trading electricity
       approving loan applications
       fraud prevention
       predicting bankruptcy
       time series forecasting
      What are Artificial Neural
 ArtificialNeural Networks (ANN) are
  biologically inspired and attempt to build
  computer models that operate like a
  human brain
     These networks can “learn” from the data and
      recognize patterns
            Basic Concepts
            of Neural Networks
 Biological   and artificial neural networks
     Neurons
      Cells (processing elements) of a biological or
      artificial neural network
     Nucleus
      The central processing portion of a neuron
     Dendrite
      The part of a biological neuron that provides
      inputs to the cell
            Basic Concepts
            of Neural Networks
 Biological   and artificial neural networks
     Axon
      An outgoing connection (i.e., terminal) from a
      biological neuron
     Synapse
      The connection (where the weights are)
      between processing elements in a neural
Basic Concepts
of Neural Networks
Basic Concepts
of Neural Networks
    Relationship Between Biological
     and Artificial Neural Networks
 Soma – Node
 Dendrites – Input
 Axon – Output
 Synapse – Weight

 ANNs    typically have much fewer neurons
    than humans
            Basic Concepts
            of Neural Networks
 Backpropagation
      The best-known learning algorithm in neural
      computing. Learning is done by comparing
      computed outputs to desired outputs of
      historical cases
 Network    structure (three layers)
     Input
     Intermediate (hidden layer)
     Output
Basic Concepts
of Neural Networks
          Basic Concepts
          of Neural Networks
Network information processing
     Inputs
     Outputs
     Connection weights
     Summation function (combination function)
     Transformation function (activation function)
             Basic Concepts
             of Neural Networks
       Loan Processing Example
        Inputs – income level, age, home ownership
        Outputs – approval/disapproval of loan nodes
             Basic Concepts
             of Neural Networks
       Connection weights
        The weight associated with each link in a
         neural network model
        strength of the data transferred between layers
         in the network
        They are assessed by neural networks learning
               Basic Concepts
               of Neural Networks
       Transformation function (activation function)
         maps the summation (combination) function onto a
          narrower range ( 0 to 1 or -1 to 1) to determine
          whether or not an output is produced (neuron fires)
         The transformation occurs before the output reaches
          the next level in the network
         Sigmoid (logical activation) function: an S-shaped
          transfer function in the range of zero to one –
       Threshold value is sometimes used instead of
        a transformation function
         A hurdle value for the output of a neuron to trigger the
          next level of neurons. If an output value is smaller
          than the threshold value, it will not be passed to the
          next level of neurons
Basic Concepts
of Neural Networks
Basic Concepts
of Neural Networks
           Learning in ANN
   Learning algorithm
    The training procedure used by an artificial
    neural network
   Supervised learning
    A method of training artificial neural networks in
    which sample cases are shown to the network
    as input and the weights are adjusted to
    minimize the error in its outputs
Learning in ANN
               Learning in ANN
       How a network learns
        Backpropagation
         The best-known supervised learning algorithm
         in neural computing. Learning is done by
         comparing computed outputs to desired
         outputs of historical cases
                 Learning in ANN
       How a network learns
        Procedure for a learning algorithm
         1. Initialize weights with random values and set other
         2. Read in the input vector and the desired output
         3. Compute the actual output via the calculations,
            working forward through the layers
         4. Compute the error
         5. Change the weights by working backward from the
            output layer through the hidden layers
Learning in ANN
   Error calculation and weights
At each hidden node and target node: compute:
   Linear combination function: C = w0 + w1x1 +…+ wnxn
   Logistic activation function: L = exp(C)/(1+exp(C)
At the target node compute Bernoulli error function: sum
   errors over all observations, where the error is -2 ln (L) if
   there is a response, or -2 ln (1 – L) if there is no
In the first iteration, random weights are used
In subsequent iterations, the weights are changed by a
   small amount so that the error is reduced
The process continues until the weights cannot be reduced
         Developing Neural
         Network–Based Systems
       Data collection and preparation
         The data used for training and testing must include
          all the attributes that are useful for solving the
         Recall the bankruptcy prediction problem we
          modeled using logistic regression -- The same data
          can be used to train a neural network:
         •   working capital/total assets (WC/TA)
         •   retained earnings/total assets (RE/TA)
         •   earnings before interest and taxes/total assets (EBIT/TA)
         •   market value of equity/total debt (MVE/TD)
         •   sales/total assets (S/TA)
         Developing Neural
         Network–Based Systems
       Selection of network structure
        Determination of:
         1.   Input nodes
         2.   Output nodes
         3.   Number of hidden layers
         4.   Number of hidden nodes
        For the bankruptcy problem (and all of our
         examples) we have one hidden layer
         •    The Bankruptcy problem has ten nodes in the
              hidden layer – sometimes one might experiment with
              the number of nodes
Developing Neural
Network–Based Systems
         Developing Neural
         Network–Based Systems
       Learning algorithm selection
        Identify a set of connection weights that best
         cover the training data and have the best
         predictive accuracy
       Network training
        An iterative process that starts from a random
         set of weights and gradually enhances the
         fitness of the network model and the known
         data set
        The iteration continues until the error sum is
         converged to below a preset acceptable level
         Developing Neural
         Network–Based Systems
       Testing
         Black-box testing
          Comparing test results to actual results
         The test plan should include routine cases as
          well as potentially problematic situations
         If the testing reveals large deviations, the
          training set must be reexamined, and the
          training process may have to be repeated
         Might compare ANN results with other methods
          such as logistic regression
         Developing Neural
         Network–Based Systems
       Implementation of an ANN
        Implementation often requires interfaces with
         other computer-based information systems and
         user training
        Ongoing monitoring and feedback to the
         developers are recommended for system
         improvements and long-term success
        It is important to gain the confidence of users
         and management early in the deployment to
         ensure that the system is accepted and used
Developing Neural
Network–Based Systems
         Neural Networks In
       SAS Enterprise Miner 5.3
 In your bankrupt project, create a new
  diagram called bankrupt_neural
 Drag the bankrupt data node onto your
 From the Model tab, drag the Neural
  Network node onto the diagram and
 Connect the data node to the Neural
  Network Node
Highlight the Neural Network node. In the property panel window, set
model selection criterion to average error
In the Property Panel window, click on the square to the right of network and
change the defaults for the Target Layer Combination, Activation, and Error
functions as indicated. Note that we are using the default of 3 hidden units
The results show an excellent fit with the cumulative lift equal to the
best cumulative lift, no misclassifications, and an average error nearly
In the Property Panel click on the box to the right of Exported Data to see the
individual predictions and probabilities. The logistic activation function at the
target level provides the probabilities, like those obtained from logistic
Similar to what we did with logistic regression add the bankruptscore
data node and the score node to the diagram as shown.
After running the score node, the output shows that 6 firms are
predicted to go bankrupt (vs. 4 under logistic regression)
For details about the individual predictions, highlight the Score node
and on the left-hand panel click on the square to the right of Exported
Data. Then in the box that appears click on the row whose Port entry is
Score. Then click on Explore.

To top