ATROCO

Algorithms for realistic image synthesis have numerous applications, for example in cultural heritage, special effects for movies, virtual prototyping and video games. A common theme for all these applications is that they require more and more realistic images, with more realistic models. But these realistic models carry a high complexity, as they are usually defined by a polygonal mesh with a very large number of polygons. A consequence of this complexity is that they are hard to represent: it takes a professional modeller several days or weeks to create a 3D model of an existing complex object such as a statue. In addition to the geometric model complexity, the reflectance field of the object also needs to be precisely represented. Generally, textures are used to this end. However, realistic textures turn out to be extremely complex, as they define a six-dimensional function, representing a bi-direction reflectance property on each individual surface point. Together, geometry and reflectance raise significant problems in terms of storage and real time processing. Our key idea in this research project is to revolutionize the classical modelling pipeline for realistic image synthesis: instead of having a person create a polygonal model of an existing object, we want to create the model directly from the object. Although several tools exist for creating digital models of existing objects, they currently require constant input from expert users, carefully controlled lighting and a long post-processing step. The models created are high-definition polygonal models, unsuitable for real-time display on standard graphics cards, and the appearance of the object, its colours, materials and textures must be captured separately by another process. Our goal is therefore to simplify the whole process of model acquisition and rendering: - Develop an acquisition method that requires little user input, and can therefore be used by novice users. This acquisition method must capture both the geometry and the appearance of objects. - Post-process the models created by the first step to make them suitable for real-time display, using a combination of geometric and image-based representation. - Render these models and integrate them into virtual worlds. This requires illumination simulation on these complex models. The sheer complexity of the models makes rendering them a very difficult task. Simulating the propagation of light inside these models and their interaction with a virtual environment is even more difficult. These three tasks are more inter-dependent than they appear at first sight: for example, the acquisition of the geometry of the models makes assumptions on the material of the model and on the lighting. Being able to conduct an illumination simulation on the model will validate these assumptions and help in creating a more accurate model. Our research project has several applications. The main application we envision is with cultural heritage: both for digital archival of items and for displaying these items with the public. But our project also has potential uses with all the applications that require the presence of a real object inside a virtual world, such as augmented reality, special effects and video