Alex Pentland, Ali Azarbayjani, Nuria Oliver and Matt Brand,
Real-Time 3D Interpretation of Human Behavior,
In The 10th Scandinavian Conference on Image Analysis,
Lappeenranta, Finland, pp. 189-196, June 1997.
Alex Pentland, Ali Azarbayjani, Nuria Oliver and Matt Brand,
Real-Time 3D Interpretation of Human Behavior,
In The 10th Scandinavian Conference on Image Analysis,
Lappeenranta, Finland, pp. 189-196, June 1997.We describe a system for real-time 3-D tracking and classification of human behavior using only modest computational resources. The system is based on use of 2-D blob features, a technique for characterizing the probability distribution function (PDF) of the appearance of interesting objects. From these features we recover 3-D shape, translations, and orientations of hands/head/mouth and the relative orientation of the cameras. The system is self-calibrating and can track people's head and hands with RMS errors of 1-2 cm in translation and 2 degrees in rotation. Patterns of behavior (e.g., hand or face gestures) can then be classified in real-time using Hidden Markov Model (HMM) methods, importantly including the new Coupled HMM methods that we have recently developed (Brand, Oliver and Pentland 1997). Typical classification accuracies are near 100%.