Suzanna Becker,
Mutual Information Maximization: Models of Cortical
Self-Organization,
Neural Computation,
7, 1, pp. 7-31, February 1996.
Suzanna Becker,
Mutual Information Maximization: Models of Cortical
Self-Organization,
Neural Computation,
7, 1, pp. 7-31, February 1996.Unsupervised learning procedures based on Hebbian principles alone are successful at modelling low level feature extraction, but are insufficient for learning to recognize higher order features and complex objects. In this paper we explore a class of unsupervised learning algorithms called Imax (Becker & Hinton, 1992) that are derived from information-theoretic principles. The Imax algorithms are based on the idea of maximizing the mutual information between the outputs of different network modules, and are capable of extracting higher order features from data. They are therefore well suited to modelling intermediate to high level perceptual processing stages. We substantiate this claim with some novel results for two signal classification problems, as well as by reviewing some previously published results. Finally, Imax and several related approaches are evaluated with respect to their computational costs and biological plausibility.