Augmented Reality as a Paradigm for Visualizing Information

Augmented reality (AR) is a technology in which a user's view of the real world is enhanced or augmented with additional information generated from a computer model. The enhancement may consist of virtual artifacts to be fitted into the environment, or a display of non-geometric information about existing real objects. AR allows a user to work with and examine real 3D objects, while receiving additional information about those objects or the task at hand. By exploiting people's visual and spatial skills, AR brings information into the user's real world. That is, AR allows the user to stay in touch with the real environment. This is in contrast to virtual reality (VR) in which the user is completely immersed in an artificial world and cut off from the real world. In VR systems, there is no way for the user to interact with objects in the real world. Using AR technology, users can thus interact with a mixed virtual and real world in a natural way. AR systems bring the computer to the user's real work environment, whereas VR systems try to bring the world into the user's computer. This paradigm for user interaction and information visualization constitutes the core of a very promising new technology for many applications. However, real applications impose very strong demands on AR technology that cannot yet be met. Some of such demands are listed below.

In order to combine real and virtual worlds seamlessly so that the virtual objects align well with the real ones, we need very precise models of the user's environment and how it is sensed. It is essential to determine the location and the optical properties of the viewer (or camera) and the display, i.e., we need to calibrate all devices, combine all the local coordinate systems centered on the devices and objects in the scene in a global coordinate system, register models of all 3D objects of interest with their counterparts in the scene, and track them over time when the user moves and interacts with the scene.

Realistic merging of virtual objects with a real scene requires that objects behave in physically plausible manners when they are manipulated, i.e., they occlude or are occluded by real objects, they are not able to move through other objects, and they are shadowed or indirectly illuminated by other objects while also casting shadows themselves. To enforce such physical interaction constraints between real and virtual objects, the AR-system needs to have a very detailed description of the physical scene.

There may be many useful applications of this technology. Just as the personal computer has changed the daily routine of the average office worker, computer technology will, in the future, very likely create even more dramatic changes in the construction, design and manufacturing industries. In order to get some idea of what this change will entail, let us examine how a typical construction worker may do his job in the future.

The Plumber of the Future

(Originally by Douglas S. Greer)
Suppose that in the year 20052015, a certain company would like to remodel a room to install a kitchen facility. To install the necessary hot and cold water pipes a plumber is contracted and visits the company's facility to make the required plans and cost estimates.

The "plumber-of-the-future" arrives carrying a small computer and after discussing where the new kitchen will be placed, he opens his personal computer case and removes what appear to be a slightly oversized pair of sunglasses. The sunglasses are connected by a wire to the personal computer and when he puts them on they act exactly like normal sunglasses; that is, he sees a darkened version of the room around him. However, after typing a few keys on his computer, when he looks around again, in addition to the room he sees a superimposed computer graphics rendition of many normally invisible objects. He sees yellow lines inside the walls representing the location of power lines and green lines that show where the telephone lines run. He sees representations of the various steel and concrete support structures inside the walls as well as the location of the heating and air-conditioning ducts.

After looking around the room he cannot see what he is looking for and so he types a few more keys on the computer. He is now able to look through the walls and see the layout of the adjacent rooms and now he finds what he is searching for: two thick blue and red lines which represent the main hot and cold water pipes running through the building. At this point he picks up a pointing device that is also attached by a wire to his personal computer. Since the physical pointing device cannot be projected through a physical wall, he sees a computer generated "extension" to the pointer which can be used to indicate positions inside the wall. Using this "extended" pointer, he now plans a path from the hot and cold water pipes to the proposed kitchen. After evaluating his plan, he makes a few modifications to simplify the installation of the new pipes and then has the computer produce a cost estimate.

After getting a final approval, he transmits the plans to the computer at his workshop where all necessary pipes are pre-cut and assembled. The next day, one of his coworkers arrives with the pre-fabricated parts and, using a similar computer setup, reviews the plan and does the installation.

Some other applications can be found in the Augmented Reality Applications Page.

The research results at ECRC are given at the Augmented Reality research results web page. For more recent results and MPEG video clips look at the page Augmented Reality (AR) Application Scenarios. A number of papers and technical reports were published on this subject at ECRC which could be found in the web page: Augmented Reality Related Publications.

Adapted from ECRC web pages.

Last Modified on 01/07/15.