Using Artificial Neural Networks for Solving Complex Problems
Faculty of Information Technology
University of Moratuwa
Abstract: During last few decades Artificial Neural modern problems are discussed. Session five
Networks have established themselves as theoretically provides some limitations with ANNs and session six
sound alternative for traditional algorithmic approach of discuss some future directions of it.
problem solving. Because of its remarkable ability to draw
meaning from complex and incomplete data, it has become
popular in solving wide range of complex problem with lot 2. An Overview
of parameters and interdependencies. Neural networks are This session gives brief introduction to what is
being used in wide range of problem areas such as health artificial neural networks, main characteristics of
care, finance, manufacturing, environment and
neural networks and also discusses about what makes
telecommunication. Purpose of this paper is to review
how Neural Networks has been used to solve real world real world problems complex and difficult to model.
problems of various areas, their success and limitations.
Areas of possible improvements and future direction of 2.1 Artificial Neural Networks
ANN are also mentioned.
Artificial Neural Networks are a different approach
of problem solving which processes the information
1. Introduction in a similar way the human brain does. Neural
A RTIFICIAL Neural Networks are been used for
solving wide range of real world problems
successfully during last few decades. Recently it has
networks resemble the human brain in two ways; (a)
its ability to acquire knowledge through learning, (b)
knowledge is stored within inter-neuron connection
taken lot of momentum with the recognition of strengths known as synaptic weights .
business community. Lots of successful applications
have been developed by the research community as An ANN consists of large number of simple
well as business world to solve various complex processing units called neurons clustered in to
problems ranging from Image Processing to Financial number of layers . These artificial neurons
Forecasting. simulate the four basic functionalities of the natural
neuron, namely receiving inputs from sources,
This review mainly focuses on how Artificial Neural combining them in some manner, performing a
Networks have been used to solve complex modern general nonlinear operation on the result, and then
world problems. Why ANN is best for solving these outputting the final result .
problems over traditional approaches and the reasons
for its success are also discussed. Limitations and
difficulties encountered when using ANN for
different problem domains and how to improve the
performance of these ANN based applications also
Rest of the paper is arranged as follows. Session two
provides a brief overview on theoretical side of ANN
and some characteristics of complex problems.
Session three discuss what makes ANN better for
solving complex problems. In session four
applications of how ANN has being used for solving Figure 1: Basics of Artificial Neuron
Most of the ANN applications are at least having be either supervised or unsupervised. There is large
three layers of neurons; input, hidden and output. number of learning algorithms available for network
Depending on the complexity of the problem number training. A back-propagation algorithm is the most
of hidden layers can vary. Simple network popular of all.
organization of an ANN is shown in Figure 2.
2.2 Complexity of real world problems
In today’s world computer applications are required
to solve more and more complex problems which
have lot of interdependencies and uncertainty. There
are some key points which make the solving of
modern problems complex and very difficult; high
number of parameters, high number of dependencies,
uncertainty of measurement data, incomplete
information, unknown input/output data and
character of parameters .
Figure 2: A simple Neural Network
When the number of parameters and their
There are six main characteristics in ANN interrelations increase, solving such problems
technology; network structure, parallel processing requires lot of computational power. Traditionally we
ability, distributed memory, fault tolerance ability, used algorithmic or statistical method to solve
collective solution and learning ability . problems. But due to the sheer complexity of these
problems, either we cannot use traditional approaches
Network Structure: Neurons in the ANN are or they are inefficient in coming up with a solution.
organized in to a layered networks structure which
can either be recurrent or non-recurrent. Recurrent 3. Use of ANN for solving complex
network is a Feedback network that calculates its problems
output based on input and feed them back to modify ANN technology can be applied to solve many of the
inputs. A non-recurrent network is a feed forward problems which were traditionally solved using
where data flow happens only in one direction, from algorithmic or statistical methodologies. In addition
inputs to output. to that, ANNs can be used to solve complex problems
which do not have an algorithmic solution or
Parallel Processing Ability: A neuron in an ANN is available solutions are too complex to found. This is
an independent processing unit. Computations mainly due to its remarkable ability to draw meaning
required to simulate ANNs are mainly matrix ones, with partial or incomplete data and six characteristic
and the parallel structure of the interconnection described earlier.
between neurons facilitates such calculations.
Using ANN input data can be mapped to output
Distributed Memory: In ANN, information is stored without known “a priori” relationship between them.
throughout the network structure rather than in one This is achieved through training.
Developments in faster computer hardware have
Fault Tolerance Ability: Parallel processing ability made neural networks a very popular choice in
and distributed memory makes ANN fault tolerance. modeling of sophisticated real world phenomenon.
Even if few parts fails in the network it do not
completely break the system, rather it will only 4. Application areas of ANN
degrade the accuracy of the output. Neural networks have been successfully applied to
broad spectrum of data-intensive, process intensive
Collective Solution: In an ANN output depends on complex applications. Applications of ANN can
the collective output of all connected neurons. If the broadly be classified in to 4 categories based on their
processing stops in the middle, partial answer will be functionality; (a) Classification, (b) Function
useless for the user. Approximation, (c) Time Series Prediction and (d)
Data Mining. Applications such as medical diagnosis
Learning Ability: What makes ANN so special is its systems, character and speech recognition systems
ability to learn from example data and use that and fraud detection systems are coming under
knowledge to solve actual problem. This learning can classification. Process modeling, process control,
data modeling and machine diagnosis are examples reliable assessment of the progression of a tumor.
for function approximation. As examples for time Clinicians are collecting lot of data about the patients,
series prediction we can specify applications such as but they are finding it increasingly difficult to process
financial forecasting, bankruptcy prediction, and and interpret them in to useful information that could
sales forecasting. Clustering, data extraction and data help them to treat patients better. Researchers have
visualization are good examples for Data mining found out that ANN can be used in this kind of
application using ANN . situations to analyze the collected data and asses the
condition of the cancer more effectively .
ANNs are being used in large verity or fields to solve
complex problems involving huge number of Esteva et al. have experimented with the use of ANN
parameters and which are difficult to solve using in predicting the outcome of postsurgical lung cancer
traditional algorithmic approaches. Mainly ANN patients . For that experiment they had used a
applications are used in fields like computer vision, sample of 67 patients who had been followed up for
health care, manufacturing, finance, decision making, at least five years after treatment. Out of that, 30
marketing, etc. How ANN has being used in these patients were alive and free of disease(FOD) and 37
fields will be described now. and recurrences or had succumbed to the disease
(REC/DEAD) . They had used feed forward network
4.1 Health care with 12x12x1 architecture and output of the ANN
In the field of health care ANNs have been used for identified two possible outcomes; free of decease or
clinical diagnosis, image analysis and interpretation, will suffer recurrences or succumbed to the disease.
signal analysis and interpretation and drug discovery. Results of the research had being successful and
Especially clinicians are faced with increasing network had being able to classify all test cases
amount of data generated through various medical correctly.
tests and biopsies, but they are finding it increasingly
difficult to interpret them in a useful way and use Angus et al. have proposed a method to predict the
them for diagnosis of disease . This session survival rate of patients suffering from breast cancer
dicusses few applications of ANN which help using neural networks . During their research they
clinicians to do take diagnosis and treatment had used the patient outcome data of patients after 72
decisions. month of the first examination, which included some
prognostic markers and lymph node involvement.
4.1.1 Clinical diagnosis of cancer using Results of the study showed the accuracy of 55% -
84%. That was 10% over the traditional approach,
ANN which considers the tumor grade and size only.
Cancer has become one of the main reasons for the
human deaths in the world. Main problem with Papnet is a commercial Artificial Neural Network
treating cancer is that most of them are diagnosed too application used for screening of Pap (cervical)
late, after it has spread all over the body. Unfortunate smears . If cervical cancer can be detected early
factor with this is that most of the cancers can be they have almost 100% chance of cure. Traditionally,
cured 100% if discovered early. To identify cancers Pap smear testing relies on the human eye to look for
clinicians have to analyze large set of information. abnormal cells under a microscope. Since a patient
Survival analysis techniques used by clinicians today with a serious abnormality can have fewer than a
are based on univariate, multivariate, and dozen abnormal cells among the 30,000 - 50,000
proportional hazards paradigms and they are not normal cells on her Pap smear, it is very difficult to
capable of doing a successful prognostic assessment detect all cases of early cancer by this "needle-in-a-
. Researchers have been able to use ANN haystack" search. When the screening is done
techniques for the diagnosis and survival analysis of manually it has very high false negative rate around
various types of cancers such as Breast cancer [1,25] 20% - 40%. By using Papnet can identify abnormal
Lung cancer  Oral cancer , and Malignant or cancerous cells with 97% accuracy, thus
Melanoma (Skin cancer) . dramatically reducing the error rate of false negatives
and potentially saving lives by accurate early
In cancer management surgeons and pathologists detection.
have to determine the existence and the extent of
spread of the cancer based on biopsies and or
auxiliary assessment at the expense of patient
morbidity and health service resources. In order to
carryout proper treatments it is essential to have a
4.1.2 Cell recognition through image optimal ways of forecasting stock market, mostly
processing and ANN those are based on assumptions and publically
In traditional method, identification of cells is being available information such as economic variables,
done using manual microscopic screening. This exchange rates, industry and sector specific
method has lot of limitations and it is quite expensive information, and individual corporate financial
since it require trained expert to do the identification. statements. During a research done by David , it
There has been lot of attempts to identify cells using was found that data selection is one of the most
image processing techniques. Most of these methods important things when using ANN for financial
used morphological segmentation, feature extraction forecasting. Finding correct set of variables to be
and classification techniques to identify cells. Since used as inputs do have a huge impact on the final
histological images contained lot of debris and prediction. So for this reason he had first used “an
synthesis material it is very difficult to do the inductive learning decision tree algorithm that
identification process using these traditional image integrates an information gain analysis technique
processing methods. with a dimension based data analysis” to select best
financial factors to be used as inputs to the ANN.
Zheng et al. have proposed a method using RGB David has experimented stock market forecasting
pixel data as inputs to the ANN rather than using with three types of neural networks (a) Multilayer
morphological data . Using this method, they feed-forward neural network, (b) Generalized
had achieved very high accuracy in cell regression neural network, and (d) Probabilistic
identification. neural network and found out that out of those three
regression and probabilistic neural networks give
Madsen et al. had successfully used image processing better results than feed forward network.
with neural networks for the classification of skin
lesions . It had been used to identify Malignant Fulcher et al. have also found out that multilayer
Melanoma a form of deadly skin cancer, which arises feed-forward back propagation networks have not
from cancerous growth in pigmented skin lesions. performed well with financial times series modeling
They have being able to detect 73.2% ± 1.9% of and prediction due to; (a) “Their activation functions
benign lesions and 75.0% ± 2.4% of malignant have fixed parameters only (e.g., sigmoid, radial-
lesions. basis function, and so on)” and (b) “They are capable
of continuous function approximation only; MLPs
are unable to handle discontinuous and/or piecewise-
4.2 Finance sector continuous (economic) time-series data” . They
Finance is very dynamic and unpredictable field. It is have used new kind of neural network called higher-
known to the researchers for long time that there are order neural networks (HONNs) for financial time
patterns in financial time series data and those data series predictions. As a result of their study’s Fulcher
are characterized by nonlinearities, discontinuities, et al. have developed different types of HONNs
and high-frequency multi-polynomial components namely Polynomial (PHONN), Trigonometric
. Market traders take decisions based on their (THONN). By using these two networks they have
previous considerations, and gut feelings; which being able to simulate and predict financial time-
make the quantification of decision making more series data with more than 90% accuracy rate.
difficult. Most of the currently available models are
based on statistical models. But due to the
complexities and non-linear nature of the financial
4.2.2 Financial distress and bankruptcy
time series data most of these models have being less prediction
effective. As a result many studies have being done In volatile financial markets investors only now that a
to how ANN can be used to predict financial time company is in trouble after they have published their
series data [14, 10]. Stock market forecasting [14, financial statements. Ability to accurately predict
10], financial distress prediction , price forecasting these situations has a significant impact on lending
[38, 6], bankruptcy prediction and credit scoring  decisions and the profitability of financial institutions
are some examples of applications of ANN in this [8, 37]. Chen and Du have done research on financial
field. distress prediction (FDP) on the companies listed in
Taiwan stock exchange corporation (TSEC) using
4.2.1 Stock market forecasting ANN and data mining techniques . They have used
Ability to do accurate forecasts about stock market is back-propagation network algorithm to discover the
a very supportive for the professionals in that field. rules and predict the FDP. Financial data of 68
For many years researcher were taking efforts to find companies listed in TSEC form January 1999 to
October 2006 had being used for this research. Out network with actual prices and found to be
those 68 companies 34 were in financial distress reasonable and accurate.
situation and other 34 were non-bankruptcy firms.
They have used 37 input variables categorized under “The term structure of interest rates is a relation of
six major types; earning ability, financial structure the yield and the maturity of default free zero-coupon
ability, management efficiency ability, management securities and provides a measure of the returns that
performance, debt-repaying ability, and non-financial an investor might expect for different investment
factors. Chen and Du have experimented with input periods in a fixed income market” . Bose et al.
data by dividing them in to 4 classes based on time; have used ANN techniques to forecast the term
past 2, 4, 6, and 8 seasons. From their research they structure of interest rates using yield curve
have found out that closer we get to the time of the forecasting . They have tested the yield curve
actual financial distress, the more accurate the forecasting using two types of neural networks; multi
prediction will be. They have achieved accuracy rate layer perceptron (MPL) and feed-forward network
of analysis for 2 seasons before the financial distress (FFN). As inputs, data of different bonds maturing at
occurs is 82.14%, and 60% for over 8 seasons. They various time intervals from Indian National stock
have also done same analysis using clustering data exchange had being used. Results of this research
mining approach and the results showed that BPN showed that MPL had mean square error (MSE) of
was always better that clustering results. 7.377 and FFN had MSE of 4.094. Average
percentage error of MPL was 0.00709 and in FFN it
Tsai and Wu have done research on bankruptcy was -0.00023. Results of this study showed that FFN
prediction and credit scoring for three datasets; performs better than MPL, in term structure of
Australian credit, German credit, and Japanese credit interest rates forecasting.
using ANN . They have used single three-layer
back-propagation network as baseline classifier to 4.3 Art and Music
compare with multiple neural network classifiers for There have being lot of research happing in how to
this experiment. From this experiment Tsai and Wu use ANN in fields such as art and music which
wanted to find answers for two questions; (a) “do involves huge amount of creativity. Most of these
multiple neural network classifiers outperform the researches are focused on allowing computers to
single best neural network classier in terms of learn and recognize musical styles, genres, or even to
predication accuracy based on a number of datasets compose music .
for the three datasets?”, and (b) what kind of neural
network classifier provide the lowest prediction 4.3.1 Music classification
errors in terms of Type 1 and 2 errors?. Tsai and Wu Malheiro et al. have done a research to classify
consider Type 1 error as “the rate of prediction errors classical music into five genres; opera, choral, flute,
of a model, which is to incorrectly classify the bad violin and piano using ANN techniques .
credit group into the good credit group” and Type 2 According to them this has been more challenging
errors as “the rate of prediction errors of a model to since the music classes that they have chosen had lot
incorrectly classify the good credit group into the bad of similarities. In this research they have used six
credit group”. They have achieved better results second segments of each musical piece with 22kHz
using single ANN classifier 97.32%, 78.97%, and sampling and 16bit quantization. Total of 40 features
87.94% respectively for three datasets than using (both temporal and spectral) have being extracted
multiple classifiers. But when Type 1 and 2 error from each sample. Classification was done using
rates were compared both single and multiple three layered feed-forward network with Levenberg-
classifiers and similar results. Marquardt algorithm. Three separate experiments
have being done to classify opera-choral, flute-violin-
4.2.3 Price forecasting piano and opera-choral-flute-violin-piano. Malheiro
ANNs are increasingly being used to forecast price of et al. have achieved 85% accuracy in flute-violin-
various things. Vahidinasab et al. have done a study piano classification, 90% accuracy in opera-choral
to see how ANN can be used to forecast day-ahead classification and 76% accuracy in flute-violin-piano-
electricity prices in Pennsylvania, New Jersey, and opera-choral classification.
Maryland (PJM) markets . They have used a
three-layered feed-forward neural network trained by Fasel et al. have used ANN to measure the tone
the modified Levenberg–Marquardt (LM) algorithm quality of Clarinet . They have used fourier
to forecast next-day electricity prices. Previous analysis to preprocess samples, producing an auditory
electricity prices and demands are used as inputs in image which is then presented to a self-organizing
this research. They had compared the results from the
map (SOM) for classification. The SOM they have by many researchers and found out to be weak from
used was a 10x10 hexagonally connected and was the cryptographical point of view . They have
constructed using the SOM toolbox for Matlab. They found out that this scheme is vulnerable to both brute
have also connected the response vectors of the SOM force attacks and known/chosen plain text attacks.
to the elements of the training set were then used as
the training set for a multi-layer perceptron (MLP) 4.4.2 Intrusion detection system
network with four output neurons to represent four With the advancements of information and
qualities of the sound; Good Tone, Closed Throat, communication technology, Intrusion Detection
Low Tongue and Low Air Pressure. Results of the Systems (IDS) have become an important part of
study have being tested using leave-one-out cross- computer networks. Bhaskar and Kamath have done
validation and were found to be 93% accurate. a research to test the efficiency of using ANN in IDS
. In their research they had used feed-forward
4.3.2 Music composition neural network using genetic algorithm based
Eck and Jurgen have used Long Short-Term Memory learning mechanism for intrusion detection in a
(LSTM) recurrent neural networks to compose music network. Dataset generated via a simulated U.S Air
. As training data they have used a form of 12-bar Force LAN at Lincoln Labs of Massachusetts
blues, popular among bebop jazz musicians with Institute of Technology have being used for testing.
quantization step size of eight notes per bar. In this Dataset consisted of 41 attributes and final output
research they have done two experiments (a) learning was either 0 (normal) or 1(malicious). Single layer
only chords, (b) learning melody and chords. Their feed-forward network with 41 input and one
LSTM networks have successfully learnt the global output nodes with supervised learning is used for
structure of a musical form and used that information the experiment. Results of their experiment showed
to compose new pieces in that same form. average 80% with best of 92% and worst of 64%
accuracy in detecting malicious connections.
4.4 Computer security
Wide spread use of computer technology in all They have also experimented with data mining
aspects of people’s lives had created a lot of technique called “Rough set” with same input
emphasis and concern over the security aspect of dataset. Results of that method showed overall
computing. There have being lot of research going on efficiency of 80% and detected a malicious
to find out possible uses of ANN technology in connection in 83% of the cases, but the deviation is
computer security field. In this session I look in to very high (30%).
some of such applications in areas of cryptography
and intrusion detection. Hybrid model of Rough Set and Neural Networks
called Rough-Neuro also being experimented by
Bhaskar and Kamath. In that method dataset is first
4.4.1 Cryptography reduced using rough set theory and that reduced
There have being attempts to use artificial neural dataset is then presented to the NN. Results of that
networks for cryptography. Yue and Chiang have hybrid model showed average of 79.4% with best of
proposed an ANN based approach for visual 86% accuracy.
cryptography . Traditionally visual cryptography
was done using codebook based approach. In their Method Best Average Worst Std. dev.
paper Yue and Chiang had proposed a semipublic 1 94.00 80.00 62.00 11.160
encrypting scheme for visual cryptography using the 2 92.00 80.40 64.00 8.356
Q’tron neural-network (Q’tron NN) model which is 3 86.00 79.60 74.00 3.627
a generalized version of the Hop eld NN model. Table 1: Performance comparison of IDS .
Their proposed scheme consists of one public share
and one or more user shares. The public share According to overall results rough-neuro method had
displays the open information; secret information outperformed the other methods with respect to the
need to be distributed is hidden inside this public worst case efficiency and standard deviation.
share. Taking the public share with different user
shares (keys), different superposed images (different
confidential information) can be viewed. 4.5 Telecommunication
Recently there have being lot of research going on to
Li et al. have also experimented on visual see the feasibility of ANN in different aspects related
cryptography system based on chaotic neural to telecommunication. This session describes such
networks . But this scheme had being analyzed uses of ANN in the field of telecommunication.
ad hoc network . In their protocol audio
4.5.1 Antenna array calibration streaming rate control mechanism is driven by
Antenna arrays are used in various communication, ANNs. Using ANNs at the source and the destination
radar and instrumentation systems. When these of a multi-hop audio stream, streaming rates are
antenna arrays are used in real world they face major adjusted among discrete values. ARC uses a set of
limitation since they require precise calibration. “It is fully connected feed-forward neural networks with
well known that to perform accurate Direction of sigmoid activation function, one for each of the
Arrivals (DOA) estimation using algorithms like response variables; throughput, end-to-end delay, and
MUSIC(MUltiple SIgnals Classi cation), antenna jitter. In their experiment they have used three
array data must be calibrated to match the theoretical possible ANN architectures; (a) ANN with neurons
model upon which DOA algorithms are based” . for each input factor and single output neuron
Bertrand, Grenier and Roy have proposed a novel (Simple Linear ANN), (b) ANN with input neurons
approach for antenna array calibration using neural for each of the factors, a hidden layer consisting of a
networks . In their study they have experimented single neuron, and a single output neuron (Simple
with three types of ANNs; ADAptive LInear Neuron Nonlinear ANN) and (d) ANN with inputs for each
(ADALINE) network, Multilayer Perceptrons (MLP) factor, interaction term, and intercept, a single hidden
and Radial Basis Functions (RBF) network. The neuron, and an output neuron (Interaction Nonlinear
antenna array they have used for this was comprised ANN).
of eight horn antennas operating in the X band (8–12
GHz). Results of their study showed that ADALINE
provided better or similar results on average to any
other conventional calibration technique while being
easy to implement in real-time. Results of other two
networks MLP and RBF were “less attractive due to
the presence of more peaks in the pseudo-spectrum
than the real number of sources”.
4.5.2 Cellular handoff
In wireless cellular communication handoff
algorithms are used to decide when and which base Figure 3: The three experimented topologies .
station (BS) to handoff in order to provide the service
uninterrupted. Hysteresis method is the most As input variables for ANNs they have used node
common handoff algorithm used. In that method speed (S), the data rate (D), and the number of end-
handoff occurs when the difference of signal strength to-end flows (F) that exist at any given time. Out of
received from target and current BS’s is higher than these three topologies Interaction Nonlinear topology
hysteresis level. Because of fading effect, the showed the better throughput and McClary et al. have
difference can be fluctuated for brief periods of time selected that ANN to be used with their ARC
which results in unnecessary handoff technique . protocol. Finally they had compared the performance
Suleesathira and Kunarak have proposed an ANN of ARC protocol with user datagram protocol (UDP)
based handoff technique . In their research they and real-time transport protocol (RTP) without
have experimented using Radial Basis Function control protocol, and a discrete variant of TCP-
(RBF) network and Back-propagation (BP) network. Friendly Rate Control (TFRC) protocol. Results of
As inputs to the networks they have used received the study showed that ARC wastes significantly less
signal strengths, direction of mobile moving to the bandwidth by reducing loss between 55% - 95%. The
target cell, and traffic intensities of the neighbor loss rate of ARC is reduced by 70% – 90% and its
cells. Results of their study showed that RBF based goodput was between 75% - 83% which was
handoff method had better performance than BP and higher than all other protocols.
Hysteresis. RBF had the lowest handoﬀ rate,
blocking rate and dropping rate. 4.6 Environment
Environment is filled with very complex phenomena.
4.5.3 Audio streaming Uses of traditional algorithmic based techniques are
McClary et al. have developed an adaptive transport not efficient if not impossible to model these
protocol called Audio Rate Cognition (ARC) for complex natural phenomena. Neural networks are
media streaming that uses ANNs to adapt the audio best suited for these kind of situations because of its
transmission rate to changing conditions in a mobile ability of learn from sample data, generalize them
and apply them to respond to unseen input data and
predict required output. This session reviews how radiation calculation using ANN technology along
ANN being used to model environmental phenomena with digital terrain models (DTM) . For their
such as air pollution, solar radiation and algal research they have used the global solar irradiance
dynamics. values recorded at 12 stations situated in the north
face of the Sierra Nevada Mountains, close to the
4.6.1 Air pollution prediction town of Hue´neja (Granada). In that study they have
Air pollution has become a major issue in many large used three layer multilayered perceptron network.
cities around the world, hence ability to predict the They have experimented with various numbers of
air quality have become very important. Gautam et hidden neurons ranging from 3-30 and achieved
al. have proposed an ANN based approach to predict maximum performance with 14 hidden neurons.
air pollutant concentrations . In their study they During their study they have being able to predict the
have compared the performance of traditional radiation values with RMSE of 6.0%. Therefore
multilayer perceptron network trained with back- according to them “that a simple method based on
propagation algorithm and with a new ANN proposed artificial intelligence is able to provide acceptable
by them which is based on the nearest neighbor results”.
search rule that helps in predicting the future state of
the time series using the weights of the nearest Rehman and Mohandes have done a research to
neighbor state. In that new approach the procedure of estimate Global Solar Radiation (GSR) using air
training the weights is similar to the standard back- temperature and relative humidity as inputs using
propagation, only in testing and prediction purpose, ANN . For their experiment they have used the
the procedure get changed. They have adopted this data on air temperature, relative humidity, GSR, etc.,
new method because of the chaotic nature of the air collected since 1998 to 2002 in a city called Abha,
pollution time series data. For this study Gautam et Saudi Arabia. Data of 1462 days during 1998–2001
al. have used the air pollutions details from city of were used for training purpose, and data for 240 days
New Delhi form months of July and August 2005. from the year 2002 were used for testing purposes.
Results of this study showed that in terms of mean During their research they have tested three NNs with
absolute percentage errors (MAPE) and normalized different input variable to see which perform best.
root mean square errors (NRMSE), the new First, they have tested a feed-forward ANN (2-24-1)
suggested method show considerable improvement to estimate GSR based on the daily maximum
when compared with results of standard back- temperature and day of the year. They have tested
propagation method. In addition to than number of another feed-forward ANN (2-32-1) with inputs of
hidden neurons required for learning was less in daily mean temperature and day of the year. Finally
proposed new method; standard back-propagation they have used feed-forward network (3-24-1) trained
provided better results with 11 (5-11-1) and 16 (6- on the day of the year, daily mean relative humidity
16-1) hidden neurons and new scheme provided and daily mean temperature to predict the GSR.
better results with 7 (5-7-1) and 13 (6-13-1).
According to them main drawback of proposed new Method Mean Squared Absolute Mean
approach is that it require more computational effort Error(MSE) Percentage Error
compared to standard MLP with back-propagation. 1 2.823 x10-4 10.3%
2 0.0052 11.8%
4.6.2 Solar radiation estimation 3 3.0148x10 -5 4.49%
Solar energy is a good source of alternative energy.
Especially in rural areas where power grids cannot Table 2: Performance comparison of GSR estimation
reach, solar energy can be used instead of burning .
fossil fuel. But in order to implement such solar
energy systems, radiation data of that location have to Results of the studies as shown in Table 2 showed
be studied. “Measure of solar radiation is usually that neural networks are well capable of estimating
accomplished by means of radiometric station nets GSR from temperature and relative humidity.
with a low spatial resolution. To estimate the
radiation in sites located away from the stations, 4.6.3 Forecasting of algal dynamics in
different interpolation/extrapolation techniques may lakes
be used” . But when the terrain becomes complex Talib et al. have successfully used recurrent artificial
calculating radiation using traditional methods do not neural networks (RANN) to forecast algal dynamics
provide accurate results. As a solution for this of two Dutch lakes called Veluwemeer and
problem Bosch, Lo´pez and Batlles have proposed a Wolderwijd . In their research they have
experimented with single lake model and merged ANNs are recognized as an efficient way of doing
lake model. In single lake model they have used 11 fault diagnosis of machinery.
years of water quality data of the two lakes from
1976 to 1993 to train RANN. For testing they have Wu et al. have researched with Probabilistic Neural
used 3 years data that represented different Networks (PNN) for fault diagnosis in internal
management periods: 1978, 1985 and 1993. In combustion engines . They had decided to use
merged lake models, 22 years data have being used PNN mainly due to its fast training capability and
for training and for testing six years of merged test ability to automatically organize the network
data (1978, 1985 and 1993) from both lakes were structure based on input data. In their research they
used. The RANN architecture they used consisted of have used Mitsubishi gasoline direct injection (GDI)
single hidden layer and hyperbolic tangent activation engine as the experimental platform and fault
function. Since they wanted to achieve five day ahead diagnosis was done based on sound emission and
forecasting the network was trained with input vibration signals of the engine. For this experiment
variables time lagged by five days. In their research they had considered six operating conditions of the
they have achieved five-day-ahead forecasts of the engine; engine without fault, intake air leakage, one
observed trends of “Oscillatoria” in Lake cylinder miss firing, two cylinders miss firing, fault
Veluwemeer with R2 = 0.87 and in Lake Wolderwijd in camshaft sensor and fault in engine coolant
with R2 = 0.66. temperature sensor. As inputs total of 75 groups of
data had collected for each operating condition.
4.7 Industrial applications
There are lots of successful applications of ANN in
manufacturing industry, especially in areas like fault
diagnosis. This session reviews few of such
4.7.1 Prediction of temperature profiles in
a vertical thermosiphon reboiler
In industries like chemical and petroleum, vertical
thermosiphon reboiler is the most widely used
reboiler type. Hakeem et al. had done a research on Figure 4: Signal-flow graph representation of fault
using ANN to predict the temperature profiles and diagnosis system .
temperatures at various operating conditions in a
vertical thermosiphon reboiler . For their research During this research they have also used traditional
they have used four layered feed-forward neural back-propagation network (BPN) and radial basis
network trained using back-propagation algorithm. function network (RDF) with proposed PPN to
For this study they have used temperature profiles of compare the performance. Result of the study (Table
three submergence level (100%, 75% and 50%). To 3) showed that PNN had outperformed BP and RBF
find optimal network architecture they have in both training time as well as recognition rate.
experimented with multiple hidden layer
configurations (10x10, 30x10, 60x20 and 20x60). BP RBF PPN
Network with 10x10 neurons in the hidden layer was Average 93.79 95.08 95.73
found to give better results than others. The values recognition
predicted from NN and experimental data were rate (%)
compared for all the three-submergence level and Training 71.734 15.0592 0.0622
found to be giving comparable answers. Results of time (s)
this study showed that ANN can satisfactorily predict Table 3: Experimental results comparison of
the temperatures profile and temperatures of a recognition rate and training time .
vertical thermosiphon reboiler.
Yang et al. have also done research on fault diagnosis
4.7.2 Fault diagnosis of machinery of rotating machinery using ANN . They have
In order to increase reliability and to reduce the loss proposed a new type of ANN which is a combination
of production due to machine breakdown, fault of adaptive resonance theory (ART) and the learning
diagnosis of machinery is very important. Fault strategy of Kohonen neural network (KNN). In
diagnosis systems can also prevent serious damages traditional NNs once it is trained new training
happening to expensive machinery. Increasingly scenarios cannot be added in to it; rather network has
to be retrained, which is very time consuming. This is
a big issue in the field of fault diagnosis because it is
very difficult to compose the training dataset
representing the features of all faults. Their concern
was to develop NN that can expand the knowledge
continuously without loosing previous knowledge
when learning new knowledge. As a result they had
come up with new network architecture called ART–
KNN that combines ART with Kohonen’s learning
strategy. To test their proposed network, they had
used machine fault simulator. Using the simulator
they had collected input data for total of eight Figure 5: Structure of proposed system .
conditions; four types of bearing faults (inner race,
outer race, ball and multiple), two misalignments
(parallel and angular), one unbalance and one 4.8 Construction field
normal condition. During data gathering they had ANNs have being successfully applied to solve
collected 32 features and later they had reduced that various problems related to construction field. In this
to eight using a parameter evaluation technique. session two of such applications; vibration control of
Result of the proposed ART-KNN was then building and predicting relative crest settlement are
compared with self organizing feature maps (SOFM), discussed.
learning vector quantization (LVQ) and radial basis
function (RBF) networks. The diagnosis success rate
for the ART–KNN was 100%, while the rates of
4.8.1 Vibration control of building
SOFM, LVQ and RBF networks were 93%, 93% and structures
89%, respectively. It showed that ART-KNN can be During last few decades there have being lot of
used very successfully for fault diagnosis. research on vibration control in civil engineering
structures. However, most of proposed control
Later Yang et al. had proposed improvement for their algorithms were based on a mathematical model of
ART-KNN by integrating it with case based the structural dynamics. Because of this mathematical
reasoning (CBR) . When solving a new problem, approach they cannot be used for structures with
the NN is used to make hypotheses and to guide the unknown dynamics. As a solution for this problem
CBR module in the search for a similar previous case Madan had proposed an ANN based approach for
that supports one of the hypotheses. The knowledge active control of earthquake-induced vibrations in
acquired by the network is interpreted and mapped building structures using counter-propagation neural
into symbolic diagnosis descriptors, which are kept network (CPN) . In his research he had
and used by the system to determine whether a final considered an eight-story building with identically
answer is credible, and to build explanations for the constructed story units to test the feasibility of
reasoning carried out. They have experimented this unsupervised learning by the CPN controller for
new approach using fault diagnosis of an electric vibration control of multi-story buildings. The
motor. They had used data from 64 cases for seven building had an active mass driver (AMD) installed
different motor defects: 14 cases with bearing faults, at the top of the building to accomplish structural
20 cases with rotor damages, 5 cases with stator control.
faults, 9 cases with air-gap related, 6 cases with
misalignment, 3 cases with mechanical unbalance
and 7 cases with components looseness. Results of
the study showed 96.9% accuracy in diagnosing.
compared the results of ANN with the result of
traditional settlement calculation theory called
Model Statistically value for error
between measured and predicted
µ σ RMSE
ANN 0.070 0.077 0.036
Clements theory 0.233 0.123 0.256
Table 4: Value of average, standard deviation, and
RSME for each prediction model 
Results of that study (Table 4) showed that even with
a data set of limited size was used in the ANN model
to predict the RCS of CFRDs, it can predict results
with highly satisfactory match with the
measurements, more than the deterministic methods.
5. Limitations of ANN
Artificial neural networks are undoubtedly a
Figure 6: Block diagram for simulation of training powerful tool for problem solving. But there are lots
and operation of CPN network neural controller for of weaknesses in this technology as well. Most of
vibration control of structures  these limitations are related to the network training,
network topology and lack of explanations for the
Results of the research showed that the CPN behavior of ANN and how to approach on solving
controller with the unsupervised learning can be more complex problems .
effectively trained to reduce the earthquake-induced
vibrations in building structures with modest Inability to explain solution
control requirements. This is the most important weakness in ANN. When
an ANN is trained to solve a problem there is no way
to verify that the network has really learnt the
4.8.2 Prediction of relative crest problem correctly and giving answers, or it is doing
settlement of concrete-faced rock ll dams some kind of cheating. If the network is doing some
Concrete-Faced Rock ll Dams (CFRD) are popular kind of cheating as happened with Pentagon project
construction type for dams around the world. to identify tanks hiding behind trees , network
Estimating and predicting how relative crest will not deliver what it promised. This verification is
settlements (RCS) and deformations of these dams really important ANNs are used in unpredictable and
happen is very important in dam construction. Crest dynamic fields like finance.
settlement is traditionally calculated using empirical
formulas. Researchers have found out that the values Finding optimal network architecture
calculated using the empirical formulas are When a network is to be designed to solve problem,
significantly different from observed values. Kim and figuring out what kind of network architecture will
Kim had done a research to predict crest settlement give the optimal solution is a big issue. Only way to
values using ANN . For their study they have find out that is to experiment with multiple network
used 30 databases of field data from seven countries architectures and find the optimal one. Even with that
(Australia, Colombia, Brazil, Thailand, Sri Lanka, kind of trial and error experiments we cannot assure
Korea, and China). They have experimented with that the architecture we chose is the optimal one;
number of different three-layer back-propagation there may be a better solution which we have not
NNs with different input variables and hidden experimented with.
neurons to find out optimal solution. After many
experiments they have found out that a NN with the Issues with training
architecture of 3-4-1 and inputs as dam height, void There are few limitations related to network training
ratio, and vertical deformation modulus after as well. Main issue related to training is that network
construction provided the best results. They have also
taking a lot of time to train, and sometimes network of fields where ANN is being applied which are
will not converge at all. In order to overcome these totally different from traditional view of ANN usage.
problems we may have to experiment with the
network by changing parameters involved in training In the fields I have reviewed, the problems where
such as learning rate and individual parameters that ANNs are being used to solve are characterized by
belong to specific learning algorithm we are using. high complexity, multivariable dependencies and
availability of unknown dynamics. Because of this
6. Future directions of ANN complex nature of the problems they are very
Few decades of research in to the field of human difficult if not impossible to model in to some kind of
nervous system has discovered lot of interesting mathematical formula. Use of NN to solve this kind
features and functions. Most of the future trends on of complex problems have being very successful in
ANN are based on these findings . most of the fields, but there is lot of room for
Hardware based NN
Almost all of the applications developed using neural Before using ANN it is also important to analyze the
networks are software implementations. Implanting input data. When the number of components in the
neural network based applications on dedicated NN input increases, ANN require more processing and
hardware has number of advantages; (a) can take most importantly ANN will use some less significant
advantage of their inherent parallelism and run orders input parameters to decide on output which may lead
of magnitude faster than software simulations, (b) to wrong results. In many applications I came across
can provide self-contained, physically robust where large number of input parameters are involved,
solutions for application areas where it might not be they have first used some kind of data mining
feasible to install a PC/workstation running neural mechanism to identify most important parameters
network software (e.g: toys, robots ). Currently there and then had used those selected inputs to the ANN.
are some hardware implementations of NN such as
pRAM-256  and Neural Network Processor According to the research papers and journal articles
(NNP) . I came across most common use of ANN is to find
patterns hidden in large quantities of data and to
Hybrid NN make estimations or predictions. And three layered
Lot of research is being done for using ANN jointly NN trained with back-propagation algorithm has
with other AI related technologies like fuzzy logic, being used to solve wide range of problems across
genetic algorithm and expert systems . multiple fields. Even though we can model most of
our real world problems using ANN with three
In most of the complex problems solved using neural layers, to achieve maximum results we have to be
networks fuzziness is a large part of the problem. For careful and have to select correct network type and
example in when automating a car stopping does not architecture. Finding this optimal solution for a
mean applying breaks at once but applying it slowly. particular problem is the most difficult issue when
To accommodate this fuzziness in to NN researchers using ANN. Current method of finding optimal
are developing new kind of neuron called “fuzzy solution through trial and error is very inefficient and
neurons” . These neurons do not simply give time consuming. More research has to be done in this
yes/no answers; rather they provide a more fuzzy area on how to find optimal ANN architecture for a
answer. particular problem.
Expert Systems (ES) cannot provide solutions for In almost all of the applications I came across during
novel situations which are not in their rule base. By this study use networks with very small number of
integrating ES with NN above situations can be neurons (less than 100). Even with small number of
handled. Once NN come up with a solution we can neurons they have performed well.
use the ES to analyze that solution and give reasoning
which is lacking in NNs. In this review I have found out that in fields like
health and finance use of ANN has being very
7. Discussion successful. Lots of commercial applications based on
In this paper I have given a brief introduction to ANN such as PAPNET can be found in those fields.
Artificial Neural Networks and reviewed about how These ANN based approaches to solve problems
they are being used to solve complex real world reduce the time and effort required considerably and
problems. In this review I have identified wide verity still provides accurate results. In areas like
telecommunication use of ANN is still in
experimental stage. There have being lot of  Bosch, J.L., Lo´pez, G. and Batlles, F.J. (2008), Daily
successful researches going on that field, especially solar irradiation estimation over a mountainous area using
in mobile communication but they are still to get arti cial neural networks, Renewable Energy, 33, 1622–
industry recognition. In areas like cryptography, use 1628.
of ANN looks promising, but they are still in very
primitive stage. Attempts to develop cryptography  Bose, Sumit Kumar, Sethuraman, Janardhanan and
scheme using chaotic networks have not being as Raipet, Sadhalaxmi. (2006), Forecasting the Term Structure
successful as expected. of Interest Rates using Neural Networks, Neural networks
in finance and manufacturing, 124-138.
Even though ANN gives correct answers to
problems, main limitation with them is that ANN  Buzzanca, Giuseppe. (2006), Music and Neural
cannot explain how they come in to that answer. Networks, Artificial Neural Networks in Real-Life
Therefore before using ANN based applications Applications, ISBN: 1-59140-902-0.
thorough testing have to be done to avoid
unpredictable results.  Chen, Wei-Sen and Du, Yin-Kuan. (2008), Using neural
networks and data mining techniques for the nancial
Future of ANN for solving complex problems looks distress prediction model, Expert Systems with
promising even though there are few limitations to Applications.
consider with. With the advancement of computer
technology more powerful computers are coming in  Eck, Douglas and Schmidhuber, Jurgen. (2002), A First
to the market. Because of that we will be able Look at Music Composition using LSTM Recurrent Neural
develop large and complex neural networks to solve Networks, Technical Report IDSIA-07-02, IDSIA.
even more complex problems.
 Enke, David. (2006), Neural Network-Based Stock
Market Return Forecasting Using Data Mining for Variable
Acknowledgement Reduction, Neural networks in finance and manufacturing,
I would like to thank my supervisor Miss.Indika 43-63.
Karunarathne for the comments, support and
guidance given for studying and preparing this  Esteva, H., Bellotti, M. and Marchevsky, A.M. (2001),
review paper. My gratitude also goes to Prof. Asoka Neural Networks for the Estimation of Prognosis in Lung
S. Karunananda for the guidance given in how to do a Cancer, Biomedical Engineering Series, 29-37.
research, and to Mrs. Subha Fernando for the
guidance given on the selection of the subject area.  Fasel, Ian R., Bollacker, Kurt D. and Ghosh, Joydeep.
(1999), A Neural Network Based Classifier and
Reference Biofeedback Device for Improving Clarinet Tone-Quality,
International Joint Conference on Neural Networks, 1924-
 Angus, B., Lennard, T.W.J., Naguib, R.N.G. and G.V. 1928.
Sherbet (2001), Analysis of Molecular Prognostic Factors
 Fraser, Neil., Neural Network Follies,
in Breast Cancer by Artificial Neural Networks,
http://neil.fraser.name/writing/tank/, downloaded on May
Biomedical Engineering Series, 9-21.
 Artificial Neural Network Technology,
 Fulche, John, Zhang, Ming and Xu, Shuxiang. (2006),
Application of Higher-Order Neural Networks to Financial
dowonloaded on February 10, 2008.
Time-Series Prediction, Neural networks in finance and
 Bertrand, Hugo, Grenier, Dominic and Roy, Se´bastien. manufacturing, 80-108.
(2008), Experimental antenna array calibration with
 Gautam, A.K., Chelani, A.B., Jain, V.K., Devotta, S.
arti cial neural networks, Signal Processing, 88, 1152–
(2008), A new scheme to predict chaotic time series of air
pollutant concentrations using arti cial neural network and
 Bhaskar, Tarun and Kamath, Narasimha. (2006), nearest neighbor searching, Atmospheric Environment, 42,
Intrusion Detection Using Modern Techniques: Integration 4409–4417.
of Genetic Algorithms and Rough Sets with Neural Nets,
 Hakeem, M.A., Kamil, M. and Arman, I. (2008),
Artificial Neural Networks in Real-life Applications, 314-
Prediction of temperature proﬁ les using arti cial neural
networks in a vertical thermosiphon reboiler, Applied  pRAM-256 neurocomputer with on-chip learning,
Thermal Engineering, 28, 1572–1579. http://www.clarkson.me.uk/academic/pram.htm,
downloaded on May 20, 2008.
 Hintz-Madsen, M., Hansen, L.K., Larsen, J. and
Drzewiecki, K.T. (2001), A Probabilistic Neural Network  R.Malheiro, et al. (2004), Classification of Recorded
Framework for the Detection of Malignant Melanoma, Classical Music using Neural Networks, 4th International
Biomedical Engineering Series, 141-183. ICSC Symposium on Engineering of Intellegent Systems.
 Introduction to Artificial Neural Networks,  Rabuñal, Juan R. and Dorado, Julián. (2006), Artificial
http://www.nddownloads1.com/videos/nns_and_neurosolut Neural Networks in Real-life Applications, ISBN: 1-59140-
ions/nns_and_neurosolutions.html, downloaded on May 18, 902-0.
 Rehman, Sha qur and Mohandes, Mohamed. (2008),
 Kim, Yong-Seong and Kim, Byung-Tak. (2008), Arti cial neural network estimation of global solar
Prediction of relative crest settlement of concrete-faced radiation using air temperature and relative humidity,
rockﬁ ll dams analyzed using an arti cial neural network Energy Policy, 36, 571–576.
model, Computers and Geotechnics, 35, 313–322.
 Speight, P. M. and Hammond, P. (2001), The Use of
 Li, Chengqing, et al., Cryptanalysis of a Chaotic Machine Learning in Screening for Oral Cancer,
Neural Network Based Multimedia Encryption Scheme. Biomedical Engineering Series, 55-72.
 Li, Eldon Y. (1994), Artificial Neural Networks and  Talib, A., et al. (2008), Forecasting and explanation of
Their Business Applications. Information & Management, algal dynamics in two shallow lakes by recurrent artificial
303-313. neural network and hybrid evolutionary algorithm,
Mathematics and Computers in Simulation, 78, 424–434.
 Li, Shujun, Chen, Guanrong and Zheng, Xuan.(2004),
Chaos-Based Encryption for Digital Images and Videos,  Suleesathira, Raungrong and Kunarak, Sunisa. (2005),
Multimedia Security Handbook. Neural Network Handoff Algorithm in a Joint Terrestrial-
HAPS Cellular System, Electronics and Communications,
 Madan, Alok. (2005), Vibration control of building 3, 164-174.
structures using self-organizing and self-learning neural
networks, Journal of Sound and Vibration, 287, 759–784.  The Fuzzy Neurons,
 McClary, Daniel W., Syrotiuk, Violet R. and Lecuire, NOWLEDGE-BASED/node78.html, downloaded on May
Vincent. (2008), Adaptive audio streaming in mobile ad 10, 2008,
hoc networks using neural networks, Ad Hoc Networks, 6,
524–538.  Tsai, Chih-Fong and Wu, Jhen-Wei. (2008), Using
neural network ensembles for bankruptcy prediction and
 Naguib, R.N.G. and Sherbet, G.V. (2001), Artificial credit scoring, Expert Systems with Applications, 34, 2639–
Neural Approach to Analyzing the Prognostic Significance 2649.
of DNA Ploidy and Cell Cycle Distribution of Breast
Cancer Aspirate Cells. Biomedical Engineering Series,  Vahidinasab, V., Jadid, S. and Kazemi, A. (2008),
Biomedical Engineering Series, 23-27. Day-ahead price forecasting in restructured power systems
using arti cial neural networks, Electric Power Systems
 Naguib, R.N.G. and Sherbet, G.V. (2001), Research, 78, 1332–1342.
Introduction to Artificial Neural Networks and Their Use in
cancer Diagnosis, Prognosis, and Patient Management,  Viharos, Zsolt Janos and Kemeny, Zsolt (2005), AI
Biomedical Engineering Series, 1-7. techniques in modeling, assignment, problem solving and
optimization, Artificial Intelligence Methods, 225-230.
 Neural Network Hardware, http://www.accurate-
automation.com/Products/NNP/nnp.html, downloaded on  What is a Neural Network?
May 20, 2008. http://www.neurosolutions.com/products/ns/whatisNN.html
, downloaded on December 6, 2007.
 Papnet Detects Cancer Accurately,
http://www.lightparty.com/Health/Papnet.html,  Wu, J.D., Chiang, P.H., Chang, Y.W. and Shiao, Y.J.
downloaded on April 13, 2008. (2008), An expert system for fault diagnosis in internal
combustion engines using probability neural network,
Expert Systems with Applications, 34, 2704–2713.
 Yang, B.S., Han, T. and An, J.L. (2003), ART–
KOHONEN neural network for fault diagnosis of rotating
machinery, Mechanical Systems and Signal Processing,
 Yang, B.S., Han, T. and Kim, Yong-Su. (2004),
Integration of ART-Kohonen neural network and case-
based reasoning for intelligent fault diagnosis, Expert
Systems with Applications, 26, 387–395.
 Yue, Tai-Wen and Chiang, Suchen. (2007), The
semipublic encryption for visual cryptography using Q’tron
neural networks, Journal of Network and Computer
Applications, 30, 24–41.
 Zheng, Qing, Milthorpe, Bruce K. and Jones, Allan S.
(2004), Direct Neural Network application for Automated
Cell Recognition, Cytometry, 1-9.