Neural Networks And
CS33Q – Artificial Intelligence Literature
What are Neural Networks?
An artificial neural network (ANN) is an information-processing paradigm. It is inspired by the way
in which the parallel architecture of the human nervous system processes information. The
beauty of the paradigm is the structure of the information processing system. It consists of many
highly interconnected processing elements; neurons (see diagram 1&2 adapted from Stergiou
and Siganus) which all work in unison to solve specific problems. Neural networks learn as
humans do, by example. They are configured through a learning process for specific applications,
such as, pattern recognition or data classification.
Diagram 1: The Human neuron
Diagram 2: The Artificial Neuron Models
How do Neural Networks function?
Artificial neurons are made to mimic the human brain. The defining element is the neuron; it
collects signals from a number of structures and then sends out spikes of electrical activity
through the axon which is split into thousands of branches. At the end of each branch the
synapse converts the activity of the axon into electrical effects that inhibit or excite activity in the
connected neurons. The neural network learns by changing the effectiveness of the synapses, so
that the influence one neuron has on another changes (Siganus and Stergiou). ANNs are an
idealized model of the real network.
ANNs operate in two modes, the using mode and the training mode. In the most basic sense the
network is trained by entering a large number of inputs to the system. It is then taught how to
react to each one i.e. when to fire and if so what output. In more complex problems the inputs are
assigned weights that determine the response of the system (see diagram 3 adapted from
Siganus and Stergiou). Pattern recognition is the most important characteristic of the networks.
They generally do this by contrasting and comparing the unknown input to all it had learned in
training mode then it chooses the output that is most similar.
There are two major subdivisions in ANNs feed-forward/bottom-up and feedback/interactive
networks. In the feed-forward setup, signals are restricted to travel in one direction; input to
output. This means that the output of one layer does not affect that same layer; because of this
feed-forwards are mostly used in pattern recognition. In feedback (see diagram 5 adapted from
works by the Japan atomic research institute) signals are allowed to travel in both directions.
These networks are more powerful and dynamic; their 'state' is changing continuously until they
reach an equilibrium point. ANNs are divided into layers; the input, hidden and output layers. The
input units represents the data, the activity of the hidden(output) units is determined by the
input(hidden) units and the weights on the connection between them.
Diagram 3: The McCulloch and Pitts model
Diagram 4: A section of the human Neural Network
Diagram 5: An Artificial Neural Network
What are the major advantages of Neural Networks?
Neural networks succeed where everyday computer systems produce mediocre results. They can
take extremely complicated or imprecise data; extract patterns and deduce trends that are too
complex to be noticed by humans or other computing techniques. In a very real way neural
networks can be thought of as an expert. They are capable of creating projections given new
scenarios. They are able to do this effectively because they can perform adaptive learning, self-
organization, real time operations and fault tolerance. Adaptive learning is the ability to learn how
to do tasks based on the data given for training or initial experience (Stergiou and Siganus). Self-
organization is the creation of its own representation of the information it receives during the
learning period. Real time operation is possible because neural network operations may be
carried out in parallel.
The Disadvantages of neural networks
The results neural networks return are at best a good approximation of a solution, they don’t
usually return an optimal solution and in some cases the results returned diverge. This is because
choosing the right structure of a neural network is in itself a complex problem. The present
technology cannot fully model the human brain that is, the technology does not scale up to
handling billions of neurons (Champandard).
The differences between Neural Networks and conventional computing
The major difference is their approach to problem solving. Conventionally computers use an
algorithmic approach; they would need to know all the specific steps to solve the problem.
Therefore they are able to solve only those problems that we know how to solve. In contrast
neural networks can perform tasks we don’t know exactly how to do. ANN’s are not programmed
to perform a specific task, instead very specific examples are carefully chosen to teach the
system. This approach allows the network to solve the problem by itself and so its operation can
Applications of Neural Networks
Neural networks have an extensive array of utilization in daily real world applications. They have
been successfully applied in a diverse range of fields which include finance, medicine,
engineering, geology and physics. The numerous tasks which we need to perform frequently can
be done through neural network implementations which are able to perform and execute these
actions reliably, effortlessly and intrepidly (humans are normally affected by fatigue and emotion).
Neural Networks In Medicine
Artificial Neural networks (ANNs) are being increasingly employed in the field of medicine. ANNs
are predominantly well suited to problems with a high degree of complexity and no algorithmic
solution, or the solution is far too complex to solve by traditional techniques. If ANNs are to be
exploited in the medical field then further research is necessary since medicine is such a critical
and meticulous practice. ANNs will have to be adaptable to various individuals and they should
be reliable enough so that by supervision by an expert is not necessary. Areas of Medicine which
have seen the successful implementation of this technology include drug development, patient
diagnosis, and image analysis - cardiograms, CAT Scans and ultrasonic scans (Neuroxl).
Research is also being done into the modeling and diagnosing of the cardiovascular system
(Siganus and Stergiou). The technology would model a range of health associated indices
including combinations of heart rate, the various levels of chemicals in the blood, systolic and
diastolic blood pressures as well as the respiratory rate. If comparison between this model and
the patients real time physiological measurements are performed frequently then this could
enable early detection of potentially detrimental diseases. Consequently a prognosis can be
established which enables early treatment of possibly threatening diseases. Momentous progress
has already been made relating to the detection of coronary artery disease and the processing of
EEG (electroencephalograph - the monitoring and recording of electrical activity in the brain)
signals. Cornerstones have been set in using ANNs to model and diagnose breast, prostate and
lung cancer. Research using neural networks has recently been able to diagnose breast cancer
with up to 90 percent accuracy and give a prognosis of life expectancy. The basis of prognosis on
life expectancy is done using microscopic images of the tumor, the age of the woman, patterns of
abnormality in the tumor such as the rate of cell division and the shape of its nuclei as inputs of
the system (Duncan). Primogeniture
Neural Networks in Business & Finance
In business and finance neural networks have been employed for tasks ranging from stock
market prediction, sales forecasting, customer research (such as analyzing a loan applicant) and
risk management. In the stock market for example many technical analysts are using neural
networks as a tool to forecast predictions of stock prices based on various factors which include
the past performance of stocks, technical information (liquidity ratios, earnings projections) and
an assortment of economic indicators (Statsoft). Neural networks have been successfully
implemented in areas such as customer research. They are able to make decisions on whether
loan applicants are qualified based on historical data including income, indebtedness, age,
occupation and credit history as the ANNs inputs. A neural network is advantageous in this kind
of operation since it could successfully make unbiased client selections. When trained on
historical data (inputs to the ANN) they are capable of meeting the objective of client analysis and
risk assessment (Siganus and Stergiou).
Neural Networks In Science
ANNs have also seen success in science such as in weather forecasting using “BrainMaker”
Neural Network software. Forecasting rainfall can be of great value especially for the purposes of
river control, reservoir operations, forestry interest and flash flood watches. Rainfall analysis has
been difficult with conventional computing methods due to complex, erratic relationships in data
and its effects on probability of predicting rainfall. However, neural networks have demonstrated
great promise with rainfall probability and quantity in a particular area with accuracy of up to 85%.
A hydro-meteorologist based at the National Weather Service in Fort Worth, Texas using
BrainMaker Neural Network software. The inputs to the ANN were nineteen meteorological
variables which include moisture, lift, potential energy and instability (California Scientific).
Neural networks in recent times have seen increasing usage in a diverse range of fields. Their
success thus far can be attributed to their ability to solve complex problems with obscure patterns
and trends which are difficult to solve by humans or conventional computing. This modern
approach to problem solving utilizes concepts very much similar to the neural activity in a
human’s brain. Neural networks are very reliable when they are trained properly and to
demonstrate confidence in them is their employment in life critical fields such as medicine.
Research is so far being done in how to diagnose and monitor a wide range of diseases early.
ANNs have a positive futuristic prospect based on the many successful implementations they
have had in various fields. They will continue to increase in popularity due to their exceptional
ability to mimic the human brain.
Siganos and Stergiou,
http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11/report.html, 20 Nov 2004
Japan Atomic Energy Research Institute,
Speed-Up of Reactor Core Design Optimization Process Using Artificial Neural Networks,
inisjp.tokai.jaeri.go.jp/ ACT98E/05/0501.htm, 20 Nov 2004
Champandard, Alex J,
Neural Networks in 3 Minutes,
http://neuralnetworks.ai-depot.com/3-Minutes/NN-Tips.html, 29 Nov 2004
http://www.statsoft.com/textbook/stneunet.html, 27 Nov 2004
Coronary artery disease
http://www.neuroxl.com/medicine_neural_networks.htm1, 27 Nov 2004
Zhi-Hua Zhou, Yuan Jiang, Yu-Bin Yang, Shi-Fu Chen
Lung Cancer Cell Identification Based on Artificial Neural Network Ensembles
http://cs.nju.edu.cn/people/zhouzh/zhouzh.files/publication/aim02.pdf, 27 Nov 2004
Artificial Intelligence Tackles Breast Cancer
http://www.newscientist.com/news/news.jsp?id=ns99992587, 27 Nov 2004
BrainMaker Predicts Rainfall
http://www.calsci.com/Weather.html, 27 Nov 2004