Application of non-adaptive neural networks for solving of the piecewise constant independent component analysis problem

Аuthors

Prostov Y. S.^*, Tiumentsev Y. V.^**

Moscow Aviation Institute (National Research University), 4, Volokolamskoe shosse, Moscow, А-80, GSP-3, 125993, Russia

*e-mail: prostov.yury@yandex.ru
**e-mail: yutium@gmail.com

Abstract

In this paper we consider the possibility of applying non-adaptive artificial neural networks (ANN) to solve non-stationary problems like piecewise constant independent component analysis (ICA). Usually adaptive ANNs are applied to such problems but this leads to large computational costs in comparison with non-adaptive models because of requirement to calculate higher-order statistics. The idea of the proposed method is based on the fact that while solving the piecewise constant problem by non-adaptive ANN, we can try to detect changes in problem conditions with consequent retraining of the network for compliance to the new conditions. If it is possible then we can significantly reduce computing costs on time intervals when conditions are constant.

The proposed method implies that we need to automate the retraining process. To achieve it we have to introduce some metric which describes the presence of changes in problem conditions. We should also adjust the learning rules of the ANN model. We show in this paper that piecewise constant ICA problem with Poisson-like independent components can be solved using non-adaptive model [5] with metric based on average neurons activity and original learning rules with only learning rates affected. The model obtained as a result was tested on the piecewise constant version of the «Foldiak bars» problem and has proved its efficiency.

Thus the proposed method can be applied in some cases to produce adaptive ANNs from non-adaptive models. However generalization of the obtained results to identify applicability conditions of the method as well as a general algorithm for ANN conversion requires further research. Furthermore convergence problem for generated models seems to be complicated enough due to the large number of influencing factors.

Keywords:

artificial neural network, adaptive models, independent component analysis

References

1. Khaikin S. Neironnye seti (Neural Networks), Moscow, ID «Vil’yams», 2006, 1104 p.
2. Choi S., Cichocki A., Amari S. Equivariant nonstationary source separation. Neural Networks, 2002, Vol. 15, pp. 121 — 30.
3. Hyvärinen A., Karhunen J., Oja E. Independent component analysis, New York, John Wiley, 2001, 505 p.
4. Fyfe C. Hebbian learning and negative feedback networks (Advanced Information and Knowledge Processing Series), London, Springer, 2005, 383 p.
5. Földiàk P. Forming sparse representations by local anti-Hebbian learning. Biological Cybernetics, Cambridge, 1990, vol. 64, no. 2, pp. 165–170.

Download