Refereed full papers (journals, book chapters, international conferences)

1993

@article{RKD1993,
	vgclass =	{refpap},
	vgproject =	{nn},
	author =	{Nicholas J. Redding and Adam Kowalczyk and Tom Downs},
	title =	{Constructive Higher-Order Network Algorithm That Is
	Polynomial Time},
	journal =	{Neural Networks},
	volume =	{6},
	pages =	{997--1010},
	year =	{1993},
	abstract =	{Constructive learning algorithms are important because
	they address two practical difficulties of learning in artificial
	neural networks. First, it is not always possible to determine the
	minimal network consistent with a particular problem. Second,
	algorithms like backpropagation can require networks that are larger
	than the minimal architecture for satisfactory convergence. Further,
	constructive algorithms have the advantage that polynomial-time
	learning is possible if network size is chosen by the learning
	algorithm so that the learning of the problem under consideration is
	simplified. This article considers the representational ability of
	feedforward networks (FFNs) in terms of the fan-in required by the
	hidden units of a network. We define network order to be the maximum
	fan-in of the hidden units of a network. We prove, in terms of the
	problems they may represent, that a higher-order network (HON) is at
	least as powerful as any other FFN architecture when the order of the
	networks are the same. Next, we present a detailed theoretical
	development of a constructive, polynomial-time algorithm that will
	determine an exact HON realization with minimal order for an arbitrary
	binary or bipolar mapping problem. This algorithm does not have any
	parameters that need tuning for good performance. We show how an FFN
	with sigmoidal hidden units can be determined from the HON realization
	in polynomial time. Last, simulation results of the constructive HON
	algorithm are presented for the two-or-more clumps problem,
	demonstrating that the algorithm performs well when compared with the
	Tiling and Upstart algorithms.},
}