基于图像绘制(IBR)技术综述

计算机图形学学科研讨会

基于图像绘制（IBR）技术综述（A Survey on Image-based Rendering Techniques）

石教英

浙江大学计算机学院

计算机辅助设计与图形学国家重点实验室

2005年5月

目录

一、基于图像绘制（IBR）技术定义

二、典型IBR技术演示

1、Panorama Mosaics

2、Tour-Into-Picture

3、Light Field

4、Feature-based Morphing

三、IBR技术理论基础-Plenoptic Funciton

1、An introduction

2、how to handle with plenoptic fuction

四、IBR技术发展

1、Image Matting

2、Digital PhotoMontage

3、High-Dynamic-Range Image Display

4、Plenoptic photography

五、基于图像绘制技术定义的扩展

Transform

Rasterization

(& Lighting)

?Computer Vision methods to recover models.

Image-Based Rendering

& Modeling

Rendering

& Modeling Volume

Rendering

一、基于图像绘制（IBR）技术定义

z Problems of triangle-based graphics:?Always starts from scratch.

?Millions of sub-pixel triangles.

一、基于图像绘制（IBR）技术定义

?Definition of IBR (by Sing Bing Kang)

image-based rendering techniques rely on interpolation using the original set of input images or pixel reprojection from source images onto the target image in order to produce a novel virtual view

一、基于图像绘制（IBR）技术定义

?Definition of IBR (by Cha Zhang & Tsuhan Chen of CMU) Given a continuous plenoptic function that describes a scene, image-based rendering is a process of two stages: –sampling and rendering. In the sampling stage, samples are taken from the plenoptic function for representation and storage. In the rendering stage, the continuous plenoptic

function is reconstructed with the captured samples.

二、典型IBR技术演示?Paronoma Mosaics

Demo

二、典型IBR技术演示?Tour-Into-Picture

二、典型IBR技术演示Lightfield Video

二、典型IBR技术演示Feature-based Morphing

三、IBR技术理论基础-Plenoptic Funciton

1. An Introduction

z Two ways of describing the world:

? A source description: ——The world can be described by

geometric models, texture maps, refection models, lighting

and shading models.

?An appearance description: ——The world can be described

by the dense array of light rays filling the space, which can

be observed by posing eyes or cameras in the space. The

light rays can be represented through the Plenoptic function.

z The traditional model-based rendering approach

adopts the source description method.

z The image-based rendering approach adopots the

plenoptic function to describe the world.

三、IBR 技术理论基础-Plenoptic Funciton ?IBR: An Old Story

z Plenoptic function z As pointed out by Adelson and Bergen (1991):The world is made of three -dimensional objects, but these objects do not communicate their properties directly to an observer. …The plenoptic function serves as the sole communication link between the physical objects and their corresponding retinal images. It is the intermediary between the world and the eye.

),,,,,,(t Z Y X f λφθ

三、IBR技术理论基础-Plenoptic Funciton ?7D Plenoptic Function

三、IBR技术理论基础-Plenoptic Funciton

2. How to Handle Plenoptic Function?

z Two Stages

?sampling and reconstruction of sampled signals z Two directions for simplication

?Restrain the viewing space of the viewers

z View point

z Perceptual

?Introduce some source descriptions into IBR

z Geometry

z Depth

三、IBR 技术理论基础-Plenoptic Funciton ?Restraining in View Space

z Assumption 1: Wavelength

?Constant wavelength

?RGB

?Almost all the practical representations of IBR make this assumption.

)

,,,,,,(t Z Y X f λφθ

三、IBR 技术理论基础-Plenoptic Funciton ?Restraining in View Space

z Assumption 2: Air

?Air is translucent

?Radiances along a light ray through empty space remain constant.

)

,,,,,,(t Z Y X f λφθ

三、IBR 技术理论基础-Plenoptic Funciton ?Restraining in View Space

z Assumption 3: Time

?Static scene

?Images captured at different time and positions can be used together to render novel views.

?Too large size for dynamic scene

)

,,,,,,(t Z Y X f λφθ

三、IBR 技术理论基础-Plenoptic Funciton ?Restraining in View Space

z Assumption 4: Viewpoint

?The viewer is constrained to be on a surface

?Acceptable:

z

Human eyes are usually at a certain height -level z Human eyes are less sensitive to vertical parallax and lighting changes

)

,,,,,,(t Z Y X f λφθ