In Augmented Reality (AR) the computer provides additional information that enhances or augments the real world, rather than replacing it with a completely virtual environment. One of the objectives of AR is to bring the computer out of the desktop environment and into the world of the user working with a three-dimensional application. In contrast to virtual reality, where the user is immersed in the world of the computer, Augmented Reality incorporates the computer into the reality of the user. The user can then interact with the real world in a natural way, with the computer providing information and assistance.
One application area that is currently being explored involves mechanical maintenance and repair. In this scenario a mechanic is assisted by an AR system while examining and repairing a complex engine. The system may present a variety of information to the mechanic. Annotations may identify the name of parts, describe their function, or present other important information like maintenance or manufacturing records. AR may lead the mechanic through a specific task by highlighting parts that must be sequentially removed and showing the path of extraction. The system may also provide safety information. Parts that are hot or electrified can be highlighted to constantly remind the mechanic of the danger of touching them. The mechanic may also be assisted by a remote expert who can control what information is displayed on the mechanic's AR system.
A major aspect of ECRC's Augmented Reality work is its emphasis on merging computer vision, image processing and computer graphics technology to form a new Human-Computer Interaction paradigm. Our specific technical approach involves matching the computer's virtual world to the real world, by first registering a virtual camera to a physical camera directed at the object of interest. Currently we use a video monitor to represent the user's view. Eventually the same techniques could be used in the appropriate applications to display graphics using a see-through head-mounted display, giving the user graphics superimposed on real-world objects. We then take a computer representation of the object of interest and match it spatially to the real one. Once this registration is established, we introduce tracking technology to maintain correspondence as the object or camera moves. To receive input from the user, we provide a tracked pointing device. Because the virtual world corresponds to the real world, the graphics drawn by the computer will appear to the user to be in the real world. Using this approach we can display additional information on the computer that appears in a proper context in the user's view. Queries about the real world are also produced and answered in the user's context. This approach integrates the computer into the user's working environment.
In the application described here, we explored this new query and display paradigm in a scenario, where AR is used to identify parts of an automobile engine that are pointed to by a mechanic. The user interacts with the real object in its natural setting with a pointing device monitored by the computer. As the mechanic points to a specific part of the engine, the AR system displays computer-generated lines and text (annotations) that describe the visible components or give the user hints about the object. Queries with the pointing device on the real-world object may be used to add and delete annotation tags. Since we also track the engine, the annotations move with the engine as its orientation changes. The lines attaching the annotation tags with the engine follow the appropriate visible components, allowing the user to easily identify the different parts as the view of the engine changes.
Other applications of Augmented Reality include medicine, manufacturing, maintenance and diagnostics, architecture, interior design, civil engineering, and computer-aided design (CAD). For example in medicine, AR will bring the geometric models of internal human structures acquired in medical scanners, such as MRI, CT and Ultrasound, into the operating room. Doctors will utilize this information to plan and guide their surgery. For manufacturing, AR may be used to quickly teach factory workers new assembly tasks.
Contact point for additional information:
David Breen
email: david@gg.caltech.edu
Adapted from an Article published in: ECRC Today, Dec. 1994, "Applications
and Demonstrators", page 6.
Last Modified on June 1, 1996.