Optimal Online Learning Algorithm For Faulty RBF Networks Under Concurrent Weight And Node Failure

Abstract

In neural networks, an important point is to make a trained network with certain fault tolerant ability. The most complicated fault circumstance is that various kinds of fault happen at the same time. That is, various kinds of network failure, including weight noise, weight fault, node noise, and node fault, occur in a single network. Although many batch mode fault tolerant algorithms were developed in the last twenty years, the development of online mode learning for fault tolerance is still in the premature stage. For instance, the conventional online fault injection algorithm cannot handle the concurrent fault circumstance. In this project, I propose an online learning algorithm for radial basis function (RBF) networks under the concurrent fault circumstance. I show that when the training rate μ is smaller than 2/[(1+σa2)(1+σb2)-(1-ra)(1−rb)]+(1-ra)(1-rb)maxj||h(xj)||2, then the proposed online algorithm converges, where xj's are the training input vectors, σb2 and σa2 are the variances of the multiplicative weight noise and the multiplicative node noise, respectively, ra and rb are the open fault probabilities for weight and node, respectively, h(xj) is a vector which contains the outputs of all RBF nodes. Besides, I show that when the learning rate μ tends to zero, the trained weight vector tends to the optimal batch mode solution. My experimental result verifies that the performance of my online algorithm is close to that of the optimal batch mode method.