Principles of Artificial Neural Networks. Read more Artificial Neural Networks: An Introduction Artificial Neural Networks B. YEGNANARAYANA Prof. Relation Between the Perceptron and Bayes Classifier for a Gaussian Environment . Write an up-to-date treatment of neural networks in a comprehensive, .. Particle Filter pdf probability density function pmf probability mass function bias applied to neuron k cos(a,b) cosine of the angle between vectors a and b. Artificial Neural Networks. B. YEGNANARAYANA. Profe.<;sor (Microsoft Clwir). International Institute of Information Technology. Hyderabad. Former Professor.

Author: | ENOCH CASHMER |

Language: | English, Spanish, French |

Country: | San Marino |

Genre: | Lifestyle |

Pages: | 794 |

Published (Last): | 04.09.2016 |

ISBN: | 346-5-51761-371-2 |

ePub File Size: | 26.88 MB |

PDF File Size: | 8.38 MB |

Distribution: | Free* [*Regsitration Required] |

Downloads: | 49223 |

Uploaded by: | ARLETHA |

ARTIFICIAL NEURAL NETWORKS by B. Yegnanarayana. O by Prentice- Hall of lndia Private Limited, New Delhi. All rights reserved. No part of this book. Artificial Neural Networks. B. YEGNANARAYANA Professor Department of Computer Science and Engineering Indian Institute of Technology Madras Chennai. artificial neural networks by yegnanarayana pdf free download artificial neural yegnanarayana b artificial neural networks prentice-hall of india soundofheaven.info artificial.

Activation and Synaptic Dynamics 62 This combines the competitive learning and differential Hebbian learning.. Note that in this case the test patterns are the same as the training patterns. I am indeed very fortunate to have understanding and accommodative members in my family. It can also be used by scientists and engineers who have an aptitude to explore new ideas in computing. Muhammad Ali Mazidi. This law is valid only for a differentiable output function.

Please try again later. Do not purchase if you want Verified Purchase. Not for Rookie. Do not purchase if you want to get some practical knowledge of ANN. This is for Rigorous Study and to get a theoretical foundation of Neural Nets. Well written book and covers the historical development of ANN. The author is a Ph. So, this books suited for Research students who really want to pursue a carrier in AI. This is one of the best book in ANN. All the topics are very well explained and organized logically, one could find the relation of one topic to the previous are the subject becomes interesting as we move ahead.

May god bless Yegnanarayana Sir with good health and happiness. Buy printed version. But this Kindle Book is a poor quality scanned version. I wanted to buy this book, but the free sample is a scanned copy of the printed book.

The quality is very bad. Not readable on my Kindle. If you want to buy, then definitely buy the printed versions. Tech projects.

That was 11 years ago. See all 3 reviews. Most helpful customer reviews on Amazon. The perceptron convergence theorem enables us to determine whether the given pattern pairs are representable or not. Topology Input Weights a1 Wl a.

The arrangement of the processing units. Connections can be made either from the units of one layer to the units of another layer interlayer connections or among the units within the layer intralayer connections or both interlayer and intralayer connections. Hence it is also known as a gradient descent algorithm.

This section presents a few basic structures which will assist in evolving new architectures. The units of a layer are. The equations that describe the operation of an Adaline are as follows: Weight change: In a feedback network the same processing unit may be visited more than once.

Widrow's Adaline model is that. Artificial neural networks are normally organized into layers of processing units. This law is derived using the negative gradient of the error surface in the weight space. This weight update rule minimises the mean squared error a2.

By providing connections to theJth unit in the F. Let us consider two layers F1 and F2 with M and N processing units. Thus the activation w. Whenever the input is given to F. During learning. They are called learning equations. We will discuss models of neuronal dynamics in Chapter 2. When the flow is bidirectional. In this section we discuss some basic learning laws [Zurada. Thus the operation of an outstar can be viewed as memory addressing the contents.

Basic Learning Laws 31 will be activated to the maximum extent. When all the connections from the units in F1 to F2are made as in Figure 1. Learning laws are merely implementation models of synaptic dynamics. If the two layers Fl and F2 coincide and the weights are symmetric. This network can be viewed as a group of instars. In the case of an outstar. Thus the operation of an instar can be viewed as content addressing the memory.

There are different methods for implementing the learning feature of a neural network..

During recall. Neuronal dynamics consists of two parts: These issues will be discussed in Chapter 4. The weights converge to the final values eventually by repeated use of the input-output pattern pairs.

This law requires weight initialization to small random values around w.. All these learning laws use only local information for adjusting the weight of the connection between two units. AwTa ] fiwTa a. The law states that the weight increment is proportional to the product of the input data and the resulting output signal of the unit.. This is a supervised learning law. The expression for Aw. This is also called discrete perceptron learning law. This law represents an unsupervised learning.

Basic Learning Laws 33 where fix is the derivative with respect to x. The delta learning law can be generalized to the case of multiple layers of a feedforward network. The input-output pattern pairs data is applied several times to achieve convergence of the weights for a given set of training data.

I t is a supervised learning law since the change in the weight is based on the error between the desired and the actual output values for a given input. This is same as the learning law used in the Adaline model of neuron. Delta learning law can also be viewed as a continuous perceptron learning law.

This law is valid only for a differentiable output function. In this case the change in the weight is made proportional to the negative gradient of the error between the desired output and the continuous activation value. The for any arbitrary training data set The convergence can be more or less guaranteed by using more layers of processing units in between the input and output layers. We will discuss the generalized delta rule or the error backpropagation learning law in Chapter In implementation..

In the implementation of the learning law. In this law the weights are adjusted so as to capture the desired output pattern characteristics. But the Hebbian learning is an unsupervised learning.

For a given input vector a. All the inputs are connected to each of the units Figure 1. The adjustment of the weights is given by Aw. This is a case of unsupervised learning. The unit k that gives maximum output is identified. M Delta Aw. The outstar learning is a supervised learning law.. Basic Learning Laws Figure 1. Besides these basic learning laws there are many other learning laws evolved primarily for application in different situations [: The weights in the case of instar can be initialized to any random values.

We have reviewed the features of the biological neural network and discussed the feasibility of realizing. Some of them will be discussed at appropriate places in the later chapters. But in order to achieve this. The initial weights could be set to random values. In some variations of these as in the principal component learning to be discussed in Chapter The Hebb's law and the correlation law lead to the sum of the correlations between input and output for Hebb's law components and between input and desired output for correlation law components.

The convergence and the limit values of the weights depend on the initial setting of the weights prior to learning. The convergence will naturally be faster if the starting weights are close to the final steady values. The set of training patterns are applied several times to achieve convergence. The learning rate parameter q should be close to one. The weights indhe instar and outstar learning laws converge to the mean values of a set of input and desired output patterns.

Basics of Artificial Neural Networks far. The set of training patterns need to be applied several times to achieve convergence. The perceptron. The most important issue in the application of these laws is the convergence of the weights to some final limit values as desired. Since the correction depends on the error between the desired output and the actual output.

Some key developments in artificial neural networks were presented to show how the field has evolved to the present state of understanding. What is meant by topology of artificial neural networks?

Give a few basic topological structures of artificial neural networks. Some features of these dynamics are discussed in the next chapter. Give two examples of pattern recognition tasks to illustrate the superiority of the biological neural network over a conventional computer system. An artificial neural network is built using a few basic building blocks. Explain the significance of the initial values of weights and the learning rate parameter in the seven basic learning laws.

Explain briefly the terms cell body. Describe some attractive features of the biological neural network that make it superior to the most sophisticated Artificial Intelligence computer system for pattern recognition tasks. What is the distinction between learning equation and learning law? While developing artificial neural networks for specific applications. Compare LMS. The building blocks were introduced starting with the models of artificial neurons and the topology of a few basic structures.

Compare the performance of a computer and that of a biological neural network in terms of speed of processing. We have discussed some basic learning laws and their characteristics. But the full potential of a neural network can be exploited if we can incorporate in its operation the neuronal activation and synaptic dynamics of a biological neural network.

Identify supervised and unsupervised basic learning laws. Explain briefly the operation of a biological neural network. What are the basic learning laws? What are the main differences among the three models of artificial neuron.

Determine the weights of the network in Figure P1. Design networks using M-P neurons to realize the following logic functions using f 1for the weights. Explain the logic functions using truth tables performed by the following networks with MP neurons given in Figure P1. Figure P1. Use suitable values for the initial weights and learning rate parameter. Basics of Artificial Neural Networks Problems 1. Use suitable values for the initial weights and for the learning rate parameter.

Select random initial weights in the range [O. Write a program to implement the learning laws. Problems Output Input vedor a. Write a program to implement the learning law.

Using the Instar learning law. Use a 4-unit input and 4-unit output network. Use suitable values for the initial weights. For a given input data. The trajectory depends upon the activation dynamics built into the network.

On the. In the previous chapter we have seen some models of neurons and some basic topologies. The activation dynamics is prescribed by a set of equations.

Since the steady activation state depends on the input pattern. The state will change if the input pattern changes. The set of weight values of all the links in a network at any given instant defines the weight state. Chapter 2 Activation and Synaptic Dynamics 2. The trajectory of the weight states in the weight space is determined by the synaptic dynamics of the network. The trajectory of the activation states. In a neural network with N processing units.

These rules are implied or specified in the activation and synaptic dynamics equations governing the behaviour of the network structure to accomplish the desired task.

A network is led t o one of its steady activation states by the activation dynamics and the input pattern. The model is not intended for detailed analysis of the network. Models of neural networks normally refer to the mathematical representation of our understanding and observed behaviour of the biological neural network.

An expression for the first derivative may contain time parameter explicitly. Activation dynamics relates to the fluctuations a t the neuronal level in a biological neural network. Introduction 41 other hand.

These expressions are usually simple enough although nonlinear to enable us to predict the global characteristics of the network. Hence this steady weight state is referred to as long term memory. Therefore a model of the neural network could be very complex. The purpose in this case is to capture the knowledge by the model. Throughout this chapter we use the terms models of neural networks and neural network models interchangeably. We must distinguish two situations here.

The objective of this chapter is to discuss models for activation and synaptic dynamics. We discuss activation dynamics and synaptic dynamics. The discussion on the activation and synaptic dynamics is adapted from [Kosko. In Section 2.

If the expression does not contain the time parameter explicitly. This function bounds the output signal. In this section we also discuss the distinction between the activation and synaptic dynamics models. We also discuss the equilibrium states of the networks with a specified activation dynamics. Since synaptic dynamics models lead to learning laws.

We shall review the general stability theorems and discuss briefly the issues of global and structural stability in neural networks. A brief discussion is included on the equilibrium of synaptic dynamics. In the final section we provide a brief summary of the issues discussed in this chapter. The output function fl. For the ith neuron. Let us consider a network of N interconnected processing units.

This is the familiar noise-saturation dilemma [Grossberg. Although the activation value is shown to have a large range. Thus the dynamic range of the external input values could be vely large. The input values to a processing unit coming from external sources. Thus the output is bounded as shown in Figure 2.

Activation Dynamics Models 43 a nondecreasing function of the activation value. Thus there is a limit to the operating range of a processing unit. The problem is how a neuron with limited operating range for the activation values can be made sensitive to nearly unlimited range of the input values.

If the neuron is made sensitive to smaller values of inputs. Figure 2.

This depends on the behaviour of the network in the neighbourhood of the equilibrium state. Structural stability refers to the state equilibrium situation where small perturbations of the state around the equilibrium brings the network back to the equilibrium state. We discuss models for activation dynamics starting from simple additive models and then moving to more general shunting or multiplicative models.

Global stability refers to the state equilibrium condition when both the synaptic and activation dynamics are simultaneously used. Thus xi t gives the rate of change of the activation value of the ith neuron of a neural network.

The model also should be able to learn adjust the weights while satisfymg the requirements of storage capacity and stability characteristics. The activation models are described by an expression for the first derivative of the activation value of a neuron.

In the following discussion we will assume that the weights do not change while examining the activation dynamics. It should be noted that each model takes into account a few features of the neuronal dynamics.

We also provide a discussion on the equilibrium behaviour for different models of the network. In developing models for activation dynamics. The input. Both of these types of inputs may have excitatory components which tend to increase the activation of the unit. Activation Dynamics Models 45 For the simplest case of a passive decay situation.

Ai can be interpreted as membrane conductance. The solution of this equation is given by In electrical circuit analogy. The initial value of xi is xi 0. With Ci. If we assume a nonzero resting potential. The passive decay time constant is altered by the membrane capacitance C. If the output function f z is strictly an increasing but bounded function.

We will discuss further on this point in a later section. The Hopfield model belongs to the class of feedback neural network models. For inhibitory feedback connections or for inhibitory external input. A network consisting of two layers of processing units.. Note that Ai and A. Activation and Synaptic Dynamics 46 In addition to the external input. The classical neural circuit described by Perkel is a special case of the additive autoassociative model.

N Passive decay term: Table 2. With external input BJi: Hetroassociative model: Bidirectional associative memory: Under special conditions.

Analogous to the Hopfield autoassociative memory. These are coupled first order differential equations. That is. We will first consider the saturation model. In order to make the steady state activation value sensitive to reflectance. Throughout the following discussion we Processing units Input intensities Figure 2.

Grossberg suggested an on-centre off-surround shunting activation model by providing inhibitory inputs from other input elements to the ith unit along with the excitatory input from the ith input element to ith unit as shown in Figure 2. For an excitatory external input Ii.

Thus it is possible to make the unit insensitive to random noise input within a specified threshold limit value. In that case the output signal of the unit will be zero. The following is the resulting model: The above shunting activation model has therefore an operating range of [.

It can be seen that. In order to make a unit insensitive to small positive inputs. The inhibitory sign is taken out of the weights wy. A shunting activation model with excitatory feedback from the same unit and inhibitory feedback from other units is given by where Jiis the inhibitory component of the external input. This steady state activation value is negative as long as the input reflectance value to the ith unit.

Bi] for the activation value. Equation 2. If the initial value xi 0 2. If we consider the excitatory term B. If the initial value x. The shunting model of 2. This can be viewed as a shunting effect in an equivalent electrical circuit. This can be proved by the following argument: Hence xi t 2.

Since the excitatory second term is always positive.. W B ] shows the contribution of the inhibitory external and feedback input in decreasing the activation value xi t of the unit. Thus there is a contradiction.

This can be viewed as shunting effect in an equivalent electrical circuit. The first term'oh the right hand side corresponds to the passive decay tenh. Hence x. Since the contribution due to the inhibitory third term is negative.

The probability distribution of the noise component is assumed for analyzing the vector stochastic processes of the activation states. The output signal of each processing unit may be a random function of the unit's activation value.

In such cases the network activation state and output signal state can be viewed as vector stochastic processes. Stochastic activation models are represented in a simplified fashion by adding an additional noise component to the right side of the expression for xi t for each of the deterministic activation models.

Each unit in turn behaves as a scalar stochastic process. To keep the operating range of activation value to a specified range General form: Saturation model: To restrict to an upper limit On-centre off-surround configuration: In the deterministic models.

The learning equation describing a synaptic dynamics model. Another factor is that the state update could be deterministic or stochastic. Another interesting feature of learning is that the pattern information is slowly acquired by the network from the training samples.

The most important among these is the update of the state change at each stage. The adjustment of the synaptic weights is represented by a set of learning equations. A large number of samples are normally needed for the network to learn the pattern implicit in the samples. That is why we say that we learn from examples.

Equilibrium of a network depends on several other factors also besides the activation models. The update could be synchronous. Note that in both the deterministic and stochastic models the transient due to the passive decay term is absent in the equilibrium state. The synaptic weights are adjusted to learn the pattern information in the input samples. The only way to demonstrate the evidence of learning pattern information is that.. In stochastic models. The equilibrium behaviour also depends on whether we are adopting a continuous time update or a discrete time update.

Pattern information is distributed across all the weights. A major issue in the study of equilibrium behaviour of a network is the speed at which the feedback signals from other units are received by the current unit.

Some of these will be discussed at appropriate places throughout the book. In the first place. The following are some of the requirements of the learning laws for effective implementation: Requirements of learning laws: The set of equations for all the weights in the network determine the trajectory of the weight states in the weight space from a given initial weight state.

The search depends on the criterion used for learning. In such a case. There are several learning laws in use. Synaptic Dynamics Models 53 is given as an expression for the first derivative of the synaptic weight wi. Learning laws refer to the specific manners in which the learning equations are implemented. Depending on the synaptic dynamics model and the manner of implementation. In supervised learning the weight adjustment is determined based on the deviation.

There are several criteria which include minimization of mean squared error. Categories of learning: Learning can be viewed as searching through the weight space in a systematic manner to determine the weight vector that leads to an optimum minimum or maximum value of an objective function.

All these factors influence not only the convergence of weights. Both activation dynamics and synaptic dynamics models are expressed in terms of expressions for the first derivatives of the activation value of each unit and the strength of the connection between the ith unit and the jth unit. Thus an on-line learning allows the neural network to update the information continuously.

Likewise the update of weight values may be in discrete steps or in continuous time. Temporal learning is concerned with capturing in the weights the relationship between neighbouring patterns in a sequence of patterns. In an off-line learning all the given patterns are used together to determine the weights. Supervised learning may be used for structural learning or for temporal learning.

There is no externally specified desired output in this case. The local information consists of signal or activation values of the units at either end of the connection for which the weight update is being made. Thus we can view the learning process as deterministic or stochastic or fuzzy or a combination of these characteristics.

These input. Structural learning is concerned with capturing in the weights the relationship between the given input-output pattern pairs. Unsupervised learning uses mostly local information to update the weights. Learning methods may be off-line or on-line. Randomness in the output state could also result if the output function is implemented in a probabilistic manner rather than in a deterministic manner. Unsupervised learning discovers features in a given set of patterns. This is discussed in this section.

I t is in the movement of the steady activation state that we would be interested in the study of activation dynamics.

The dynamics model may have terms corresponding to passive decay.. If there is no input. The case of synaptic dynamics model is different from the activation dynamics model. The equilibrium states x correspond to the locations of the minima of the Lyapunov energy function V x. Note that providing the same input at another instant again causes the weights to change.

We are only interested in the equilibrium stable states reached by the steady state activation values for a given input. The objective in synaptic dynamics is to capture the pattern information in the examples by incrementally adjusting the weights. Here the weights change due to input.

The equilibrium behaviour of the activation state of a neural network will be discussed in detail in Section 2. The transient part is due to the components representing the capacitance and resistance of the cell membrane. Note that even a single unit network without feedback may have transient and steady parts.

This results in a set of N coupled nonlinear equations.. Synaptic Dynamics Models 55 determine the equilibrium state that the network would reach for a given input. But in a network with feedback fiom other units. The passive decay term contributes to transients. In discrete implementation. As an example. If the model contains a passive decay term in addition to the terms due to the varying external input. This assumes that the transients decay faster than the signals coming from feedback.

The initial weight wii 0 can be viewed as a priori knowledge.. O receives importance as can be seen below fmm the solution o t t h e equation without the passive decay term. Most of the time the learning laws ignore the passive decay term.

As mentioned above..

The second term reflects recency effect. The activation values considered here are steady and stable.. The weights are expected to capture the patterns in the input samples as determined by the synaptic dynamics model. This assumption is reasonable. Then the initial weight w. The above solution shows that the weight accumulates the correlation of the Note that the activation values output signals.

The term wy 0 e" can be considered as a forgetting term.. There is an exponential weightage to this accumulation. This is because the activation dynamics depends on the external input besides the network parameters like membrane capacitance and the connection topology like feedback. It shows the accumulation of the correlation term with time. In discrete-time implementation. Categories of learning Supervised.

Competitive learning-leaniing without a teacher Linear competitive learning Differential competitive learning Linear differential competitive learning Stochastic versions. It does not decay with time. The change in the weight due to an input pattern at the time instant t is given by In summary. Learning Methods 57 summation of the correlation terms. Fundamentals of neural network modeling. Artificial Neural Networks and Information Theory.

Artificial Neural Networks in Real-life Applications. Recent advances in artificial neural networks.

Analysis and Applications of Artificial Neural Networks. Intelligent Systems: Approximation by Artificial Neural Networks.

Clinical Applications of Artificial Neural Networks. Recommend Documents. Artificial Neural Networks B. Neural Network Design Neural Network Theory Alexander I.