Application of Artiﬁcial Neural Networks and the
Genetic Algorithm to Semiconductor Device
Modelling - A Review
Koﬁ Deh∗ Sheila Prasad†
This paper presents an overview of the application of artiﬁcial neural
networks (ANNs) to the modelling of semiconductor devices and circuits.
A brief introduction to the deﬁniton of the ANN is followed by the applica-
tion to the modelling of HBTs in high-speed networks. First, we describe
the analysis of ampliﬁer circuits with ANNs. Then, we describe the ap-
plication of ANNs to the synthesis of circuits. The genetic algorithm is
deﬁned and the application of the structured genetic algorithm to device
modelling is described. It is shown that these modelling methods are su-
perior to the conventional optimisation methods in terms of accuracy and
1 Introduction - ANN theory
Optimisation is a process of determining the optimal solution to a design prob-
lem i.e.,the best possible solution according to some speciﬁed decision criteria.It
can be transformed into an iterative process which searches for improved solu-
tions to a preliminary design. The preliminary design is analysed and the results
are compared with the design speciﬁcation. Any deviation from the speciﬁca-
tion is minimised by properly adjusting the appropriate parameter values. In
circuit modelling of devices, the optimisation problem is considered to be a
curve-ﬁtting procedure. It is used to obtain the values of unknown parameters
in the equations describing the device. Regardless of the nature of the equa-
tions representing a device, a criterion for determining the best set of model
parameters has to be speciﬁed. The most commonly used criterion is the least
squares diﬀerence function which presents problems with convergence and the
computations are very tedious and time-consuming.
∗ Columbia University, New York, USA
† Professor Emeritus, Northeastern University, Boston, MA., USA.,email@example.com
2 Artiﬁcial Neural Networks
Artiﬁcial neural networks (ANNs) are based on the human nervous system which
consists of a distribution of neurons to carry messages back and forth to the
brain. ANNs have been used successfully in many modelling and optimisation
applications such as in control systems and communicatons. They are particu-
larly useful when several tasks are to be performed in parallel and computation
rates are required to be high. A comprehensive overview on ANNs has been pub-
lished by Vai and Prasad . Neural networks are particularly attractive because
of their speed and accuracy. Hence, neural networks have been developed into
an alternative computer-aided approach to model and design high speed devices
and circuits.The neural network approach features a learning process which ﬁne
tunes neural network parameters to interrelate the variables being modelled.
This approach utilises the associative capability of a neural network to globally
and concurrently evaluate the eﬀect of varying all the parameters. When used
in place of a physics-oriented device model, a neural network avoids the need to
repeatedly solve the equations that describe the device physics. Two classes of
neural networks are used in modelling:
1. Multi-layer perceptron neural networks
2. Recurrent Hopﬁeld neural networks.
The neural network consists of many processing elements called neurons,
each connected to many others. Every connection entering a neuron has a
weight assigned to it. This weight is used to amplify, attenuate, or change the
sign of the signal in the incoming connection. The input to the neural network
is a vector of the data to be modelled. Each neuron operates on the outputs of
other neurons according to its transfer function and delivers an output to other
neurons. Often, the transfer function sums the incoming signals to determine
the value of the neuron’s next output signal. The result is an output vector
representing some characteristics associated with the input.
In order to use neural network algorithms, it is necessary to determine an
interconnection pattern, the weights, and the transfer functions. The creation
and training of an appropriate neural network for the problem on hand is dif-
ﬁcult and time consuming. However, an appropriately trained neural network
provides fast and eﬃcient solutions that have shown excellent results for diﬀer-
ent applications such as the modelling of transistor behaviour and microwave
circuits as well as microwave impedance matching. A neural network consists
of a set of simultaneous non-linear equations that are capable of modelling any
continuous function when the appropriate weights are determined.However, the
problem of ﬁnding the appropriate weights for the network is a highly nonlinear
one suggesting the necessity to use stochastic optimising algorithms such as ge-
netic algorithms. We now describe the strucure of a generalised neural network.
Consider a multi-layer neural network with 4 layers ( 1 input, 2 hidden and 1
output ). Let X = (x1, ...xi ....xm ) be the input vector, G = (g1, ...gj ....gn ) the
output of the ﬁrst hidden layer, H = (h1, ...hk ....hp ) the second hidden layer and
Y = (y1, ...yl ....yq ) the output layer. In addition, uij is the weight between the
ith input and the j th neuron in the ﬁrst hidden layer; vjk is the weight between
the j th neuron in the ﬁrst hidden layer and the k th neuron in the second hid-
den layer; and wkl is the weight between the k th neuron in the second hidden
layer and the lth neuron in the output layer. Bias terms acting like weights on
connections from units whose output is always 1 can also be provided to the
neuron. The output of the neural network is computed as
yℓ = (1)
1 + e−γℓ
where γℓ is the weighted total input to the output neuron ℓ, which is deﬁned as
γℓ = hk wkℓ (2)
and p is the number of neurons in the second hidden layer. Similarly, the output
of the second hidden layer H can be expressed as a function of the output of
the ﬁrst hidden layer G which can in turn be expressed as a function of the
input vector X. The back propagation training algorithm aims to adjust the
weights of a MLP neural network in order to minimise the sum-squared error
of the network, which is deﬁned as
E(n) = (dml − yml (n)) (3)
where n is the epoch number in the training process, S is the number of training
data, q is the number of output variables, dm = [dm1 dm2 ...dmq ] and ym =
[ym1 ym2 ...ymq ] are the mth desired and calculated output vectors, respectively.
This is done by continually changing the values of the weights in the direction
of steepest descent with respect to the error function E. The iteration process
continues until the error function is minimised. The learning is performed by
many presentations of a prescribed set of training examples to the network.
One complete presentation of the entire training set during the learning process
is called an epoch.The learning process continues on an epoch-by-epoch basis
until the weights of the network stabilise and the error function converges to a
3 Structured Genetic Algorithm
Unfortunately, the problem of ﬁnding the appropriate weights for the network
is a highly nonlinear one. This suggests the need for for the use of stochastic
optimizing algorithms such as simulated annealing. Genetic algorithms, inspired
by the Darwinian theory of evolution, are another paradigm from nature that
has found applications in device modeling.
Genetic algorithms usually proceed by randomly generating several possible
solutions to a problem and evaluating how close the generated solutions are to
Figure 1: SGA chromosome. Dark colored neurons are currently inactive and don’t
contribute to network output.
the desired solution. The classic method of representing these solutions is in
the form of strings called chromosomes. Each chromosome consists of the prob-
lem variables called genes. The best solutions (chromosomes) are selected and
combined in a process called reproduction. Reproduction consists of crossover
and mutation. During crossover, pairs of chromosomes are separated at selected
points and their halves are exchanged. Reproduction will, hopefully, result in
better solutions to the problem. Mutation is a randomizatiion procedure that
usually ﬂips a gene in the chromosome. Its purpose is to keep the solution from
getting stuck in a local minima. The algorithm as described here assumes a
feedforward neural network with one hidden layer.
1. Randomly generate a population of P1 bit sets (arrays of 0s and 1s) each
with length equal to the MAX NEURONS. MAX NEURONS is the max-
imum number of neurons the hidden layer can have.
2. Initialize a population of P2 arrays with small real random numbers. The
arrays represent the weights between the input and hidden neurons. Each
member of P2 has a corresponding member in P1 .
3. For each individual, calculate the activations of the hidden neurons. Then
calculate the output layer weights by using linear regression to solve for
the “best ﬁt” of these activations to the output data.
4. Evaluate the ﬁtness of each member of P2 using the mean square error of
the solution obtained in the previous step. Each evaluation uses only the
weights corresponding to activated neurons in P1 .
5. Select individuals for reproduction based on their ﬁtness values. The se-
lection method is described in Section 3.
6. Apply the crossover and mutation operators to a fraction, pcross, of P1
and of P2 and replace the old population with the children.
7. Evaluate the ﬁtness of every member in the population.
8. If the termination condition is not met, go to step 3.
9. Select the best individual and exit.
Figure 2 gives the ﬂow chart for the structured genetic algorithm.
with L. R.
Evaluate member Increment
fitness and select generation
for reproduction count by 1
Figure 2: Structured genetic algorithm for neural networks
4 Circuit synthesis using artiﬁcial neural net-
Designing an analog circuit is a diﬃcult problem requiring many years of ex-
perience and intuition. Although, an attempt to replicate such a process using
neural networks may, at ﬁrst sight, look like a futile one, it should be remem-
bered that circuit design is a search problem in which the designer is trying
to maximize an objective function for many possible choices of circuit compo-
nents. This was the basic idea used in the application of genetic programming
to create a comprehensive circuit synthesis algorithm . This system takes as
input a number of embryonic (basic) circuits and a target optimization goal.
Using special functions, it modiﬁes each circuit by adding new components and
connections. The algorithm then creates netlists for each circuit which are
then simulated with SPICE computer to measure the electrical performance
values. A genetic ﬁtness value is assigned to each circuit based on the close-
ness of it electrical performance to the desired value. The ﬁttest circuits are
selected for cross-over and reproduction to form new circuits for the next gen-
eration(iteration) of the algorithm. The genetic programming application to
circuit synthesis has been very successful and resulted in the design of several
complex analog circuits. Although there have been similar applications of neu-
ral networks for optimization of the values of circuit elements  , none has
been as far- reaching as the genetic programming method.
In this section, we use the Hopﬁeld network to demonstrate circuit synthesis
using a neural network. The discrete-time Hopﬁeld network, shown in Figure
Figure 3: Hopﬁeld network showing 4 summing neurons,bipolar activation func-
tions and biases
3, is a model of a content-addressable memory. It can store a number of pat-
terns and retrieve the closest- matching pattern when presented with an input
pattern.The patterns are encoded into the network as the weights of neuron con-
nections. The inputs to the Hopﬁeld network must be bipolar. This is done by
converting real-valued circuit parameters to binary equivalents. The algorithm
is most easily demonstrated by using the newhop function of the MATLAB
neural network toolbox. The neural network input vector x′ is updated by the
network until its values show no appreciable change, i.e. x(k + 1) ≈ x(k). When
the network moves to this stable state, it becomes suitable for automated circuit
design. The low-frequency gain of the cascode MOS ampliﬁer shown in ﬁgure
4 was modelled in  using this method. The Hopeld network is a good tool
for illustrating the concept of using autoassociative memories for circuit design.
However, our experiments showed it is able to recall patterns correctly only
when the network size is small. Since the introduction of the Hopeld network,
other autoassociative neural network architectures have been proposed. The
principal component analysis neural network (PC-ANN) is one example . A
PC-ANN can be regarded as consisting of two parts: an output section that is
a reection of the input section. The output section reproduces the input data
Figure 4: Equivalent circuit of a cascode MOS ampliﬁer
after the network is trained. It has been shown the rst section compresses the
input data into their principal components and the output section expands the
principal components into the original data. This means the PC-ANN is capa-
ble of storing information for larger amounts of data than the Hopeld network,
which does not perform data compression. For future work we plan to replace
the Hopeld network with the PCANN architecture. For the neural network to
completely synthesize a circuit, it will have to store information about the cir-
cuits connections. The graph neural network algorithm, which was developed
for modeling structured data with neural networks, is also a promising approach
to developing a complete method for synthesizing circuits with neural networks.
This paper presents a review of recent work applying neural networks to device
circuit modelling.There are several diﬀeent avenues that can be taken to improve
the techniques and widen the applications. A complete method for synthesising
circuits with neural networks is a promising approach for the future.
 M. Vai and S. Prasad,”Neural networks in microwave circuit de-
sign - beyond black-box models”, ”Int. Journal RF and Microwave
Computer-aided Engineering”,vol.9, no.3, pp.187-197, May 1999.
 K. Deh and S. Prasad,”Microwave circuit synthesis using neural
networks”, Unpublished manuscript.
 J. Koza, F. Bennet, D. Andre, M. Keane, and F. Dunlap, Auto-
mated Synthesis of Analog Electrical Circuits by Means of Ge-
netic Programming, IEEE Transactions on Evolutionary Compu-
tation, vol. 1, no. 2, (9) pp. 109128, July 1997.
 K. Shirakawa, M. Shimiz, N. Okubo, and Y. Daido, Structural
determination of multilayered large-signal neural network hemt
model, IEEE Transactions on Microwave Theory and Techniques,
vol. 46, October 1998.
 A. H. Zaabab, Q. Zhang, and M. Nakhla, A Neural Network
Modeling Approach to Circuit Optimization and Statistical De-
sign, IEEE Trans. on Microwave Theory and Tech., vol. 43, no.
6, pp. 13491358, June 1995.
 M. A. Kramer, Nonlinear Principal Component Analysis Using
Autoassociative Neural Networks, AlChE Journal, vol. 37, no. 2,
pp. 233243, February 1991.