Acquiring 3D Shapes and Models from Image sequences of objects

Recent progresses on multimedia, information network, and virtual reality raise new requirements of modeling 3D objects. Such objects can be commercial products, industrial parts, folk arts, sculptures, environment, antiques and archaeological finds, etc. New display tools, for example VRML, have also been available for viewing objects via network. How to input 3D models of objects conveniently and stably thus becomes a challenging problem. Although various laser range finders have been developed for measuring 3D shape, they are still too expensive and object specific. We select video camera as the input device for 3D modeling because it has been widely used even in family. Our objective is: receiving a standard image sequence through network from a user or an image sequence recorded in a video tape, we construct 3D models and return the data to the user. To make reconstruction process robust, we fix the way of taking images rather than just waving the camera in the space and solving a general problem. We rotate an object around an axis and take an image sequence in the direction orthogonal to the axis. The rotation angle is controllable or measured in the images. This camera setting is easy to be achieved. It not only simplifies the algorithms of shape recovery, but also creates an intrinsic circumstance for investigating natures of different visual cues. We have studied shape recovery on each rotation plane using contour, highlight, edge, and shading according to their motions in the corresponding epipolar plane image. We explored how to combine their results for a complete model.

Shape from Moving Patterns Shape from Moving Contour Shape from Moving Highlights Shape from Moving Shading Fusing Multiple Visual Cues
Objects in the Images
Recovered 3D Models

Looking at an example of input image sequence and the Epipolar Plane Image at a selected height.

My publications