Virtual Environment Display System - ACM 1986.pdf
  Virtual Environments: Personal Simulations & Telepresence." (1991)

TELEPRESENCE Ð One application of the VIEW system was to interact with a simulated telerobotic task environment. The system operator could call up multiple images of the remote task environment that represent viewpoints from free-flying or telerobot-mounted camera platforms. Three-dimensional sound cues give distance and direction information for proximate objects and events. Switching to telepresence control mode, the operator's wide-angle, stereoscopic display is directly linked to the telerobot 3D camera system for precise viewpoint control. Using the tactile input glove technology and speech commands, the operator directly controls the robot arm and dexterous end effector that appears to be spatially correspondent with his own arm.

DATASPACE - Advanced data display and manipulation concepts for information management were also developed with the VIEW system technology. Efforts included use of the system to create a display environment in which data manipulation and system monitoring tasks are organized in virtual display space around the operator. Through speech and gesture interaction with the virtual display, the operator could rapidly call up or delete information windows and reposition them in 3-space. Three-dimensional sound cues and speech-synthesis technologies were used to enhance the operators overall situational awareness of the virtual data environment. The system also has the capability to display reconfigurable, virtual control panels that respond to glove-like tactile input devices worn by the operator.

In the Aerospace Human Factors Research Division of NASA's Ames Research Center, an interactive Virtual Interface Environment Workstation (VIEW) was developed as a new kind of media-based display and control environment that is closely matched to human sensory and cognitive capabilities. The VIEW system provided a virtual auditory and stereoscopic image surround that is responsive to inputs from the operator's position, voice and gestures. As a low-cost, multipurpose simulation device, this variable interface configuration allows an operator to virtually explore a 360-degree synthesized or remotely sensed environment and viscerally interact with its components. The Virtual Interface Environment Workstation system consists of: a wide-angle stereoscopic display unit, glove-like devices for multiple degree-of-freedom tactile input, connected speech recognition technology, gesture tracking devices, 3D auditory display and speech-synthesis technology, and computer graphic and video image generation equipment.

When combined with magnetic head and limb position tracking technology, the head-coupled display presents visual and auditory imagery that appears to completely surround the user in 3-space. The gloves provide interactive manipulation of virtual objects in virtual environments that are either synthesized with 3D computer-generated imagery, or that are remotely sensed by user-controlled, stereoscopic video camera configurations. The computer image system enables high performance realtime 3D graphics presentation that is generated at rates up to 30 frames per second as required updating image viewpoints in coordination with head and limb motion. Dual independent synchronized display channels are implemented to present disparate imagery to each eye of the viewer for true stereoscopic depth cues. For realtime video input of remote environments, two miniature CCD video cameras are used to provide stereoscopic imagery. Development and evaluation of several head-coupled, remote camera platform and gimbal prototypes was also carried out to determine optimal hardware and control configurations for remotely controlled camera systems.

The earliest device for interactivity in a virtual environment and with virtual objects was the dataglove developed at NASA Ames. Based on an invention developed by Tom Zimmerman while he was at Atari Research for measuring motion of a single finger, the gloves were custom built for NASA by Zimmerman at VPL Research and later marketed by VPL as a commercial product. These gloves were fitted with special sensors to measure the bend of the fingers and equipped with a magnetic tracking system that allowed for the glove , and the hand inside it, to be followed in 3D space and the ability to handle virtual objects freely. Special software was also developed in the VIEWlab to allow different gestures for specific actions and system commands such as "flying" through the virtual environment, interacting with virtual menus, or easily scaling models of virtual objects.

For a long time, the images seen by visitors to virtual space - such as these images produced by NASA in the 1980s - were very simple. Until recently it was very difficult and expensive to generate complex images fast enough Ñ twenty or thirty images a second Ñ for the user to have the impression of real immersion in a virtual environment, with instantaneous changes in what he saw corresponding to movements of his head and eyes. Today this technology has made considerable progress and there are now high performance computer cards that can create detailed stereoscopic images at very high frame rates. Similar devices are also becoming available for use with ordinary personal computers.

A VIEWlab Virtual World database for use in Surgical planning and education in collboration with MIT Media Lab and Stanford Medical School. The medical applications of telepresence have made rapid progress over the past few years. One notable development is a virtual cadaver on which surgeons can test new operating techniques and which can also be used for training medical students. NASA, Stanford University, and the Massachusetts Institute of Technology have designed a virtual skeleton with the tendons and muscles attached to the bones and which the user may manipulate as though in plastic surgery. Joseph Rosen, Scott Delp and Steve Pieper were the main creators of this system. NASA has also experimented with remote surgical operations to be carried out on astronauts by robots. These tests were abandoned, however, since the delay in transmitting the information represented too great a risk factor. This research was directed then towards the idea of remote medical assistance, with experts using telepresence to give advice to medical staff on board a spaceship.

VIEWlab demo(1985) Quicktime movie (20.9MB)

VIEWlab demo (1989) Quicktime movie (69.5MB)