Benny Lautrup, Lars Kai Hansen, Ian Law, Niels
Mørch, Claus Svarer and Stephen Strother,
Massive Weight-Sharing: A Cure for Extremely Ill-Posed
Problems,
In H. Hermann, D. Wolf and E. Pöppel eds., Workshop on Supercomputing in Brain Research: From
Tomography to Neural Networks, Jülich, Germany,
p. 137, HLRZ, World Scientific, November 1994.In most learning problems, adaptation to given examples is well-posed because the number of examples far exceeds the number of internal parameters in the learning machine. Extremely ill-posed learning problems are, however, common in image and spectral analysis. They are characterized by a vast number of highly correlated inputs, e.g. pixel or pin values, and a modest number of patterns, e.g. images or spectra. In this paper we show, for the case of a set of PET images differing only in the value of one stimulus parameter, that it is possible to train a neural network to learn the underlying rule without using an excessive number of network weights or large amounts of computer time. The method is based upon the observation that the standard learning rules conserve the subspace spanned by the input images.