Tracking a driver's hands using computer vision

This paper describes a computer vision system for tracking a driver's hands. The applications of such a system are discussed with particular attention given to the control of a novel vehicle systems interface. This interface is controlled by pointing at the required function on a display mounted behind the top of the steering wheel. The vision system must detect and track the driver's hands to allow efficient recovery of a search space in which to look for a pointing finger. Cameras mounted above the driver provide images of the steering wheel area. The scene is segmented using adaptive background and foreground models. A distance transform is applied to the resulting contours. A geometric model is fitted to the resulting distance map by maximising an objective function over the model parameters in a local search space defined by the output of a Kalman filter. The appearance-based model fitting process can resolve ambiguous situations such as the hands touching or crossing, and parting again. The system is demonstrated running at 10 Hz on a standard PC.