ATTENTIVE PANORAMIC SENSING FOR VISUAL TELEPRESENCE

摘要

Sensor and bandwidth constraints limit the instantaneous spatial resolution and field-of view (FOV) achievable in any visual system. In the human eye, a compromise has evolved in which resolution is high near the optical axis, but falls off with eccentricity. The effective resolution of the system is extended by fast gaze-shifting mechanisms and a memory system that allows a form of integration over multiple fixations. We have constructed an artificial visual system based on these concepts. The peripheral component consists of a catadioptric video sensor that provides a panoramic FOV. The foveal component is a video pan/tilt camera with 14 deg FOV. Calibration yields a table of projective parameters, indexed by foveal pan/tilt coordinates, that allows rapid transfer of pixels between foveal an d panoramic coordinate frames. A second transform maps between sensor and display frames. Alpha masking yields a circular, smoothly blended fovea embedded in the lower resolution panoramic image. The system may be operated in 3 modes. In slaved mode, mouse-clicks in the display generate saccade commands to the pan/tilt platform. In autonomous mode, saccades are entirely determined by motion detected in the peripheral sensor. In semi-autonomous mode these two independent motor command streams are arbitratedto produce a system responsive to operator interest as well as autonomously- detected motion events. The display duration of foveal images from past fixations is determined by a memory parameter. At one extreme, previous foveal data are immediately replaced by more recent lo w resolution data from the panoramic sensor. At the other extreme, a sequence of fixations builds up a persistent high resolution mosaic. In intermediate modes, foveal data from previous fixations gradually fade into more recent low-resolution data. The system is presently operating on a Pentium platform at 15 fps.