Refereed full papers (journals, book chapters, international conferences)

1991

We study the evolution of the generalization ability of a simple perceptron with N inputs which learns to imitate a ``teacher perceptron''. The system is trained on p = alpha N binary example inputs and the generalization ability measured by testing for agreement with the teacher on all 2^N possible binary input patterns. The dynamics may be solved analytically and exhibits a phase transition from imperfect to perfect generalization at alpha =1. Except at this point the generalization ability approaches its asymptotic value exponentially, with critical slowing down near the transition; the relaxation time is propto left (1- sqrt alpha right )^-2. Right at the critical point, the approach to perfect generalization follows a power law propto t^- frac 12. In the presence of noise, the generalization ability is degraded by an amount propto left ( sqrt alpha - 1 right )^-1 just above alpha = 1.