IVR-based computational reconstruction method

IVR-based computational reconstruction method in three-dimensional integral imagingwith non-uniform lens arrayChang-Keun Kim, Keong-Jin Lee, Dong-Choon Hwang, Seung-Cheol Kim, Eun-Soo Kim3D Display Research Center, National Research Lab of 3D Media Dept. of Electronic Eng, Kwangwoon University447-1 Wolgye-Dong, Nowon-Gu, 139-701 Seoul, Korea,xero21@kw.ac.krAbstract: One of the main challenges in 3D display and visualization is to overcome its limited depthof focus. However, this process suffers from the problem that it is to deteriorate the resolution of thereconstructed images. In this paper, we propose an IVR-based computational reconstruction method toenhance the resolution of reconstructed images in three-dimensional integral imaging with a non-uniform lenslet array having different focal lengths and aperture sizes. To show the feasibility of theproposed method, the experiments are performed and its results are presented.©2008 Optical Society of AmericaOCIS codes: (110.0110) Imaging systems; (110.6880) Three-dimensional image acquisition; (100.6890) Three-dimensional image processing1. IntroductionGenerally, the integral imaging (II) technique has been regarded as one of the most promising techniques for 3-D imaging and display [1,2], because it can provide full parallax, continuous viewing and full-color images. Although the II system has those advantages, it still has some problems including small image depth, limited image resolution and narrow viewing angle. Even though there are many advantages in II, it is difficult to reconstruct 3-D images which have large depth range in the conventional II system due to limited pickup range depending on depth-of-focus (DOF) of lenslet array. For the practical application, the extending methods of DOF have been investigated [3]. The method is using non-uniform lenslet array with different aperture and focal length. The DOF range of 3-D reconstructed image is known to be highly dependent on the aperture and focal length, so that it was extended by use of various aperture and focal length. However, this method has a problem that is decreased resolution of reconstructed 3-D images because the number of elemental images is reduced. Recently a computational integral imaging reconstruction (CIIR) method [4] to reconstruct resolution-enhanced 3-D images using the intermediate-view reconstruction technique (IVRT) has been proposed [5]. With this technique, intermediate elemental images can be digitally synthesized as many as required by using only a limited number of pickuped elemental images and from which resolution-enhanced 3-D images can be reconstructed.In this paper, we propose an IVR-based computational reconstruction method that has a lenslet array with different focal lengths and aperture sizes, which can increase the DOF and the resolution of reconstructed image. In the pickup process, the lenslet arrays with different focal lengths and aperture sizes are used to increase 3-D image depth and the IVRT is used to improve the resolution of 3-D image reconstructed by use of CIIR from the elemental image array (EIA). To show the feasibility of the proposed system, some experiments on extending DOF and improving the resolution of 3-D reconstructed images with the 3D object of toys are performed and its results are presented.2. Proposed methodFigure 1 shows an overall block-diagram of the newly proposed scheme largely consisting of three processes using a CIIR technique: pickup, generation of intermediate elemental images, and reconstruction processes. That is, in the pickup process, EIA of a 3-D object are have been captured from the pickup process of II through a non-uniform lenslet array and in the generation process of intermediate elemental images, the number of elemental images is can be increased by use of IVRT. Finally, in the reconstruction process then 3-D images are can be reconstructed from the EIA increased by use of IVRT at the various output planes by use of the CIIR technique.ComputationalPick-up CIIR IVR techniqueReconstruction 3D images 3D ObjectFig. 1. Flowchart of propose scheme3. Experiments and result In this paper, the experiment is carried out to prove the proposed algorithm. Figure2 shows an experimental setup for pickup of elemental images and display from the 3D object by using a non-uniformlenslet array. 74mm50mm DOFLIP(1)LIP(2)DOF of the nonuniform-lens array The nonuniform-lenslet arrayCCD74mm 50mmFig. 2. Experimental setup for pickup and display using the non-uniform lenslet arrayA test object is composed of a …car‟ and …hat‟. Each object has a resolution of 1200×1200 pixels. The …car‟ and …hat‟ are longitudinally located at z=50 mm and z=74 mm, respectively. The lenslet array which is composed of 24x24 number of lenslet with two kinds of characteristic is located at z = 0mm . The specifying of the non-uniform lenslet array like focal length f , aperture d and each LIP are showed table 1. And each recorded elemental image is sized 20x20, 30x30 pixels.Table. 1. Specifying of the non-uniform lenslets(unit : mm)#1 #2 F5 5.1676 d0.5 0.74And table 2 is blur measure of each reconstructed …car ‟ image. The (a) is reconstructed …car ‟ image which is located at DOF, the (b) is reconstructed …car ‟ image which is off-focusing effect by out of DOF and the (c) is reconstructed …car ‟ image by non-uniform lenslet array. This table show that the (b) is blur than the (a) and the (c) become sharper than the (b).Table. 2. Blur measure of each reconstructed …car ‟ image by each method(a) (b) (c) Blur measure 0.0324 0.05694 0.03612We apply to IVRT on pickuped EIA through non-uniform lenslet array for enhancement of resolution ofreconstructed 3-D image. The reconstructed image based on IVR has a resolution of 2370×2370 pixels, while the conventional reconstructed image has a resolution of 1200×1200 pixels, because the number of elemental image is increased by IVRT. Therefore reconstruction 3D image based on IVR has not only larger depth but also higher resolution.(a) (b)Fig. 3. IVR based reconstructed plane images by use of nonuniform lenslet array (a) …hat‟ image at z = 50mm (b) …car‟ image at z =74mmFigure 3 shows reconstructed plane images based on IVR by use of nonuniform lenslet array. In Fig. 3, the element image is increased as application IVR. Consequently the Fig. 3 is reconstructed with large depth and image of the higher-resolution.4. ConclusionIn II, the conventional method with a DOF has limited depth range and out of DOF images are blurred due to off-focusing. And the problem can be conquered from generating several DOF by the use of non-uniform lenslet array with differential focal length and aperture method. However, this has also decreased the resolution of reconstructed image by reciprocal proportion in the number of differential characteristic lenslet. Therefore, in this paper, a novel IVRT based CIIR method in II system using non-uniform lenslet array with differential focal length and aperture is proposed and experimentally demonstrated. In the proposed method, the DOF range of 3-D reconstructed image is extended and at the same time resolution of the reconstructed 3D images can be significantly improved by increasing of number of IEIs. Through some successful experiments, a feasibility test of the proposed method has been performed.AcknowledgmentThis research was supported by the MIC (Ministry of Information and Communication), Korea, under the ITRC (Information Technology Research Center) support program supervised by the IITA (Institute of Information Technology Assessment) (IITA-2007- C1090-0701-0018).References[1] G. Lippmann, “La photographic integrale,” C.R. Acad. Sci. 146, 446-451 (1908).[2] C. B. Burckhardt, “Optimum parameters and resolution limitation of integral photography,” J OSA A 58, 71-76 (1968).[3] J.-S. Jang and B. Javidi, “Large depth-of-focus time-multiplexed three-dimensional integral imaging by use of lenslets with nonuniform focal lengths and aperture sizes,” Opt. Lett. 28, 1924-1926 (2003).[4] D.-H. Shin, E.-S. Kim and B. Lee, “Computational Reconstruction of Three-Dimensional Objects in Integral Imaging using Lenslet Array,” Jpn. J. of Appl. Phys. 44 8016-8018 (2005).[5] J.-S. Park, D.-C. Hwang, D.-H. Shin, E.-S. Kim, "Enhanced-resolution computational integral imaging reconstruction using an intermediate-view reconstruction technique," Opt. Eng. 45, 117004 (2006).。