Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications Fusion of Neural Networks, Fuzzy Systems and Genetic Algorithms: Industrial Applications
by Lakhmi C. Jain; N.M. Martin
CRC Press, CRC Press LLC
ISBN: 0849398045   Pub Date: 11/01/98
  

Previous Table of Contents Next


Weight Update for Recurrent Path

The weight update equation for the recurrent path is somewhat different. For simplicity, consider unit delay (i.e., m = 1). For better clarification, the neuron in Figure 5b is redrawn with recurrent connection showing 2 parts: P1 and P2 (Figure 6). The output of P1 is In and the output of P2 is netpjhidden as described before. Thus, we have

Let fn = netpjhidden. Then, the partial derivative of fn with respect to WRj,1 becomes

Using standard gradient descent and chain rules, it can easily be shown that


Figure 6  A neuron from layer 2 (Figure 5) is redrawn showing 2 parts: P1 and P2. The output of P1 is In and the output of P2 is netpjhidden as used in Figure 5a.

5.2.2 Temporal Information and Weight Update

It is to be noted that the temporal information is accumulated by the recurrent path. Hence updates need to be done in batch mode. Thus, the above weight changes are calculated and stored for each pattern, but weight update is done only after all patterns are applied and by summing all ΔWij for each pattern. This process is then repeated until convergence is obtained. Thus, we follow schemes similar to real-time learning as reported in [7]. Accumulation of information also takes place in the “recall” mode, and it has corresponding impact on the fuzzy rules as described below. It should be noted here that the net only remembers the recent past, i.e., it gradually forgets the old context.

5.2.3 Recurrent Fuzzy Logic

Like NeuFuz, the first layer neurons (Figure 5a) include the fuzzification process whose task is to match the values of the input variables against the labels used in the fuzzy control rule. The first layer neurons and the weights between layer I and layer 2 are also used to define the input membership functions. The layer 2 neurons represent fuzzy rules. However, considering the recurrent connection (Figure 5a), the fuzzy rule format is accordingly changed to incorporate the recurrency. Thus, the recurrent fuzzy rule format is

IF input 1 is Low AND input 2 is Low AND previous output is Y1
THEN the new output is X
[recurrent fuzzy rule with one delay]

considering recurrent connection with unit delay. Y1 is a singleton like X. However, it can have two forms:

a)  Y1 same as WRj,1 and
b)  Y1 as the product of netpjhidden(t-1) and WRj,1.

Clearly, the previous output information is incorporated in the rule’s antecedent and is represented either by weights WRj,1 or by (WRj,1·netpjhidden(t-1)), although the latter makes more sense from temporal information point of view. Thus, fuzzy processing is different for these two cases, although both would essentially yield the same results.

For recurrent connection with n-delays, the recurrent fuzzy rule becomes,

IF input 1 is Low AND input 2 is Low
AND last output is Y1
AND 2nd last output is Y2
AND 3rd last output is Y3
...
...
AND n-th last output is Yn
THEN the new output is X
[recurrent fuzzy rule with n delays]

To ensure that we do not lose any information (or accuracy) in mapping neural net to fuzzy logic, a one-to-one mapping is used (for both versions of Y singletons). This approach has the key advantage of generating fuzzy logic rules and membership functions that meet a pre-specified accuracy. To ensure one-to-one mapping between the neural net and the fuzzy logic, neural network based fuzzy logic algorithms are used, as described for the NeuFuz.

Recurrency can also be used in the inputs. A typical fuzzy logic rule considering single delay would look like

IF input 1 is Low AND previous Input 1 is Medium
AND input 2 is Low AND Previous input 2 is Low
AND last output is Y1
AND 2nd last output is Y2
...y
...
AND n-th last output is Yn
THEN the new output is X
[recurrent fuzzy rule with delays at inputs and outputs]

Extension to multiple delay case is straightforward.

5.2.4 Determining the Number of Time Delays

Use of too many feedback delays complicates fuzzy rules, uses more memory, and does not necessarily improve the solution. This is because the context information is useful only when the signals in the near vicinity belong to the same membership function (i.e., when autocorrelation function does not decay rapidly). With a large number of delays in the feedback path, all distant signals will not belong to the current membership class and so the recurrent information will not be useful. Typical values for number of delays are from 1 to 5. As shown in Table 1, context information with more than one delay does not help much. Also, note that the recurrency significantly improves the convergence time.

Table 1 Number of cycles used to learn a sine wave with 9 data points. 5 membership functions were used.
Accuracy Number of cycles to converge
1D 2D 3D 4D NeuFuz (no recurrency)
0.01 9 8 8 7 1036
0.001 172 160 160 158 5438


Previous Table of Contents Next

Copyright © CRC Press LLC