In this paper we develop a group theoretic model for the feature extraction part of pattern recognition systems. We argue that the features used should reflect the regularities in the environment in which the system exists. We develop first a group theoretical model to describe these regularities, and then we show how to construct a feature extraction system that reflects these regularities. We show why the so found filter functions often appear as solutions to optimality problems and why they often have some nice properties such as invariance under the Fourier transformation. We will mainly investigate problems connected to the group of rotations (in 2-D and 3-D space) but we will touch other types of symmetries as well.