Recent Developments in 3-D Imaging Technologies

Recent Developments in 3-D Imaging Technologies

Jung-Young Son,Bahram Javidi ,Fellow,IEEE ,Sumio Yano,and Kyu-Hwan Choi

Abstract—Three-dimensional (3D)imaging technologies that have been developed recently are reviewed.In recent years,many new progresses in the 3D imaging have been made but most of them provide more practical solutions to comply with the commer-cialization efforts of the 3D imaging.Based on time-multiplexing,revisiting the stereoscopic imaging methods is one of the solutions to make possible each view image has the full panel resolution.The rapid progresses of high speed LCDs and the initiation of stereoscopic imaging service to mobile phones have made the method possible.The efforts also encouraged more concerns in evaluating 3-D image quality and identifying factors affecting the quality.In this respect,in-depth analysis has been done for the multiview imaging methods,and other forms of multiview 3D image display methods,such as the focused image/pixel array of multiview images are also analyzed.

Index Terms—Continuous parallax,focused image array,full resolution stereoscopic imaging,holographic imaging,image quality,multiview 3D,3D imaging technologies,time-multiplexing.

I.I NTRODUCTION

S

TEREOSCOPIC and multiview images are still consid-ered as the most adaptable methods to the current plane imaging systems,among the currently known three-dimensional (3D)imaging methods [1]–[6].This is why commercialization efforts have been concentrated on the stereoscopic and mul-tiview images for the last several years by the major display companies in the world.However,the full volume market for the 3D image displays is still some distance away.This is not because of the lack of 3D imaging services and contents but because the quality of 3D images is usually somewhat worse than that of plane images,which is now in the super High Def-inition (HD)level.The resolution of super HD is four times higher than that of Full HD

(19201080),which is considered as having a remarkable image quality.This is probably why major display companies are interested in stereoscopic imaging with full display panel resolution [7]–[10]more than the mul-tiview 3D.The recent development of high speed LCD which is capable of operating 240-Hz frame rate makes feasible the stereoscopic imaging.This LCD is used either as display panel

Manuscript received December 01,2009;revised March 01,2010;accepted March 02,2010.

J.-Y.Son is with the School of Computer and Communication Engineering,Daegu University,Kyungsan,Kyungbuk,Korea (e-mail:sjy4271@hanyang.ac.kr).

B.Javidi is with the Department of Electrical and Computer En-gineering,University of Connecticut,Storrs,CT 06268USA (e-mail:bahram@https://www.360docs.net/doc/403250948.html,).

S.Yano is with the National Institute of Information and Communications Technology (NICT),Keihana,Kyoto 619-0289,Japan,(e-mail:yano.s@nict.go.jp).

K.-H.Choi is with Samsung Advanced Institute of Technology (SAIT),Yongin,Gyeonggi,Korea (e-mail:kh0715.choi@https://www.360docs.net/doc/403250948.html,).

Color versions of one or more of the ?gures in this paper are available online at https://www.360docs.net/doc/403250948.html,.

Digital Object Identi?er 10.1109/JDT.2010.2045636

or parallax barrier.In the display panel,left and right eye im-ages are switched time-sequentially to display stereoscopic im-ages with each view image having full display panel resolution.The time sequential display method is especially advantageous to the displays with a very limit resolution,as in mobile phones.The stereoscopic image service for the mobile phones has al-ready been provided,but the typical resolution of LCD panel mounted on the mobile phones,

320240is too small to display the stereoscopic images in a spatial multiplexing fashion [11].The time-sequential method allows making

320240image resolution for each view image in the mobile displays.

Compared with the stereoscopic image,3D images based on the multiview images have been considered as providing more natural depth sense and comfort to viewers.But the resolution of each view image is low compared with the panel resolution and becomes lower as the number of different view images to be displayed increases.To mitigate this problem,either many low resolution projectors such as pico-projectors or many high res-olutions projectors as a focused light beam array of multiview images are used [12],[13].Using more projectors,the resolu-tion of each view image is not even further increased but also the number of multiview images.These approaches are paving to achieve a super-multiview image [1].But the cost of building them will be very high and aligning the projectors requires much more efforts than other multiview 3-D imaging methods at least for now.These problems will be aggravated more as the number of the projectors increases.Hence,multiview imaging methods [4]are at least for now not given much consideration for com-mercialization by the major display companies.The commer-cialization efforts are also urged to develop proper methods for evaluating and improving 3D image quality and re?ning prob-lems related with current 3D imaging method [14]–[17].For this purpose,activities for developing guidelines for 3D dis-plays based on ergonomics and for standardization of technical and biomedical speci?cations for the display have been initial-ized [14].In addition to the activities,an experiment to make ?t the ?at panel displays to 3D has also been performed [18].This has a signi?cant meaning because 3D displays are getting out from the subordinate relationship to ?at panel displays.In the electro-holographic imaging areas,only few new developments related with fast calculation algorithm for CGH (Computer Gen-erated Hologram)have been reported [19]–[21].But recording and displaying 3D natural scene based on fringe pattern calcu-lation with the image depth information obtained from a depth camera [22]or a multiview camera array [23]are an impres-sive new result.From these developments,it is not dif?cult to imagine that super-multiview and holography based 3D imaging methods,and new ?at panels for the methods will be the major subjects for the future 3D imaging research.

In this paper,3D imaging methods developed recently,and other developments related with existing 3D imaging methods,are reviewed.The priority given to this paper is providing

1551-319X/$26.00?2010IEEE

in-depth information regarding the underlying principles of those methods.However,the 3D imaging methods reviewed in the previous articles [1],[2]are only brie?y mentioned whenever they are necessary.

II.D ISPLAY C HARACTERISTICS OF P LANE AND 3D I MAGES In general,all plane image displaying screens and panels are diffusing the images to their front space to increase viewing space,though the diffusing angles are different for different screens and panels.Hence,when two different plane images are displayed simultaneously based on a spatial multiplexing scheme or are displayed time sequentially based on a time-mul-tiplexing scheme,they look mixed because the images are blurred as shown in Fig.1(a).There are a couple of ways to make these images be separately viewed:One is making each of these images have different optical characteristics to each other and the other to have different viewing directions,as shown in Fig.1(b).The former is for the eyeglasses type stereoscopic imaging methods,such as color ?lter (anaglyph),polarization and shuttering (transparency change),and the latter is for the multiview 3D imaging methods including non-eyeglasses type stereoscopic imaging methods [2].For the case of the former,since two images with different optical characteristics are coex-isting in the viewing space,each view image will be selectively viewed through a media which is transparent to the optical characteristic of the image.When the optical characteristic of the media composing a side of the eyeglasses is matched with that of one of the two images and the other side with the other image,the eyeglasses become transparent and the two images will be perceived as an image with a certain depth if the images are from a stereoscopic image pair.By this way the media in each side of the eyeglasses is directing the image to the eye behind the media.Hence,the eyeglasses is used to create viewing regions for separately viewing each of the two images.For the latter case,each view image is optically directed to a de?ned direction so that it is not mixed with other view images and viewed at a speci?c viewing direction.This direction will have a small range because rays composing the image are diverging.There are several ways of directing each view image to its de?ned direction,such as using a viewing zone forming optics (VZFO),a point light source array[24],a focused point array of multiview images [12],[13],a point array of multiview pixels [25]and a collimated image array[26].The VZFO is composed of an array of elemental optics.This elemental optics is typically a pinhole,a microlens,a line slit and a long cylindrical lens.The VZFO can have other forms such as grating array,Fresnel lens,holographic screen,micro-retarder .But recently,

micro-retarder lenticular [17],a varifocal lens [27]and a tunable microlens array made of liquid crystal [28],and micromirror arrays [29],[30]are also introduced.The VZFO is attached in front of the display panel to use its optical power to separate each view image from the multiview images displayed on the panel and direct it to (a)de?ned direction(s)to create its viewing region(s)in its front space.It also works as the image projection screen for projection type.However,the vari-focal lens aligns each frame image on the display panel to focus at different depths to create a volumetric image.The eyeglasses for stereoscopic images are movable forms of VZFO because viewers can move around within the viewing

space.

Fig.1.Diffused (plane)and directed (3D)images.

The micromirror arrays can be used both ways.For the ?rst application,a 45degree tilted micromirror array is fabricated on a pinhole plate.The pinhole array in the pinhole plates has the same period as the micromirror array and each micromirror can re?ect beams at its both sides.Through the pinhole array,a pixel from each view images 2and 3are alternately aligned and they are simultaneously displayed.At the same time the

micromirrors above the pinholes for view image 2

tilt

and those above image 3,.Hence view image 2is

directed direction from the normal and view image

3,.Next view images 1and 4are displayed the same as view images 2and 3.

Now the tilting angle are changed

to

.Now the view images 1and 4are directed to

a

and directions,respectively,from the normal.This is shown in Fig.2.For the screen ap-plication,the micromirror array can create viewing region for each view image by different tilting angle combinations of micromirrors in the array but it is still in concept stage.

The image and pixel array methods are very similar to each other in principle because each of their image elements is angu-larly multiplexed with a pixel from each view image composing the multiview images.The pixel position in each view image is the same as the element position in the multiview image dis-play screen/panel.The point light source array has similar op-tical structure as the image and pixel array methods,though it is a simple light source.The main difference between them is that the former creates a de?nite viewing region for each view image but the latter no de?nite viewing region.However,since each point light is only illuminating a part of the screen/panel in its front and the illuminated parts are linearly expanded with distance as the source does,it can be considered that the source is virtually the focused image of the part.The illuminated part by each point light source should be composed of at least more than a pixel cell to create a proper viewing zone.The pixel cell is a basic unit of the screen/panel of the multiview 3D imaging systems and corresponds to an element in the image and pixel array methods.As explained above,the expanded image by each point light source array has the same optical characteristics as a projector image from image and pixel array methods [13].Since the source array is composed of 2D point lights,it can generate a full parallax image while the image and pixel array methods generates a horizontal parallax only.As the source array,an LED array is used but OLED display itself can also be used.In fact recently,a line array of OLED is combined with a lentic-ular plate to be used as a line source array for a multiview 3D image display [31].

SON et al.:RECENT DEVELOPMENTS IN 3-D IMAGING TECHNOLOGIES

3

Fig.2.Multiview image display by double-sided mirror

array.

Fig.3.Recording geometry of Zebra hologram.

The quality of the multiview 3D images is related with the number of pixel cells and pixels in each pixel cell/element.These numbers increases,the quality improves.They can be hardly increased with currently available ?at panel displays but the focused point array methods can because they can use a projector array or angularly multiplexed multi-pixel point array.The number of different view images and resolution of each view image will be increased by aligning more number of and higher resolution projectors or pixel points.Though there is no convinceable evidence yet but it has been considered that increasing the number of different view images and their resolution can also display a hologram like image as shown by Zebra hologram [32].The Zebra hologram is a stereo hologram and consists of a 2-D pixilated hologram array recorded on a photopolymer.Each pixilated hologram is named as hogel and made by focusing a very high resolution image to a point as in the holographic memory [33].Fig.3shows the recording geometry of the Zebra hologram.Each of a multiview image array is displayed on a spatial light modulator and illuminated with a converging laser beam to record it as a point hologram on a photoplate.By shifting the photoplate in both horizontal and vertical directions according to the relative position of each view image in the multiview image array,a point hologram array corresponding to the multiview image array is formed on

the photoplate.The reconstructed image of the Zebra hologram is consisted of a point array of focused images.Each of the focused image point in the array will be diverging with the angle the same as the focused angle for recording.Hence the reconstructed image will not be different from the displayed image of the image and pixel array methods.Instead of using photopolymer,photorefractive polymer can be used [34].This material is rewritable and has a high density,the refreshable Zebra hologram can be recorded on the media.

III.P ROGRESSES IN S TEREOSCOPIC I MAGES

The conventional multiview 3D imaging methods including stereoscopic imaging had a critical disadvantage of sharing the resolution of a display panel with the multiview images to be displayed together for the 3D image display.Due to this sharing,the resolution of each view image is reduced to the value cor-responding to the display resolution divided by the number of different view images within the multiview images.This is one of reasons reducing the 3D image quality.To overcome this problem,full panel resolution stereoscopic imaging methods based on time-multiplexing have been developed to improve the quality of stereoscopic images to the level of plane images.The methods are especially useful for mobile phone displays which are with low resolution displays.The methods can be realized

4JOURNAL OF DISPLAY

TECHNOLOGY

Fig.4.Principle of displaying a stereoscopic image with full panel resolution.

three different ways:1)combining two high speed?at display panels as image display and VZFO,respectively;2)one high speed?at panel for image display and directional light guides, for non-eyeglasses type;and3)one high speed?at panel for image display and high speed shutter glasses as VZFO for eye-glasses type.The working principle of the high speed VZFO method is depicted in Fig.4.A high speed LCD panel for VZFO is creating a parallax barrier.For the starting,the images are interlaced with right and left image order but next the order is reversed with shifting open slit of the parallax barrier to a half period to the left as shown in Fig.4.For the next periods these two sequences are repeating.Since this sequence is re-peated more than120Hz,right and left images have the full resolution of the image panel without?ickering.For the case of the eyeglasses-type,the whole image panel displays left-right or right–left image order and a high speed LCD shutter glasses ON-OFF sequences of the right and left sides are synchronized with the image display order.In this shuttering process,to min-imize the crosstalk caused by LCD relaxation time,shutter ON time is set slightly shorter than the OFF time.For the case of using directional light guides,VZFO is not necessary because the light guide directs the illuminating lights for right eye image to the right eye direction and for left eye image to left.The light guides have been made by making grooves on the bottom sur-face of a plastic plate[35]and patterning prismatic and lentic-ular structure on a plastic?lm[36].The former case,the groove depths are increasing and periods are decreasing as the distance from the light source increases.The latter case,the guide has the structure of a lenticular sheet on the top of a light guide.The bottom of the lenticular sheet is grooved with a prism array.In these systems LED is used for the light source.Left and right LEDs lit alternately and their working periods are synchronized with the images on LCD panel.

There are several other developments in stereoscopic images.

A volumetric stereo image display[37]and non-eyeglasses-type polarization based stereoscopic displays[17],[38].The volu-metric stereo image display is providing the real image depth to the stereoscopic images by seeing each eye image through a focal length controllable lens assembly.The image displayed on the display panel imaged to the distance corresponding to the stereoscopically induced depth by controlling the focal

length Fig.5.Non-eyeglasses-type full panel resolution stereoscopic image display based on lenticular+microretarder.

of the lens assembly.In this way,the image depth becomes the same as the perceived image depth,i.e.,vergence and accommo-dation are provided.The stereoscopic displays have two exam-ples;one is using a micro-retarder plate combined with a lentic-ular plate and the other is using viewing zone limiting aperture. The former is essentially a directional light guide type stereo-scopic imaging method using polarized lights.Micro-retarder+ lenticular is functionally generating directional beams for left and right eye images.Fig.5shows the structure of the display. The lights from back light panel are polarized in a certain direc-tion.These polarized lights are passing through a polarization switch.This switch is to rotate the polarization direction of the incoming light to90when it is operating.After passing the switch,the lights are passed a strip-type micro-retarder.In this retarder,zero and half wavelength strip retarders are alternately aligned.The half wave retarder strips rotate the polarization di-rections of incoming lights90.After the retarder,the lights are passing the lenticular plate.Since the width of each cylindrical lens in the lenticular plate is slightly less than that of a strip pair, the magni?ed images of the same retarding strips are converged

SON et al.:RECENT DEVELOPMENTS IN 3-D IMAGING TECHNOLOGIES

5

https://www.360docs.net/doc/403250948.html,parison of principles of hologram and photo taking.

to space as shown in Fig.5.The zero retarder strip images are converged to the right-hand side and half wave strip images to the left side.These two retarder strip images have 90polar-ization difference.If the LCD panel polarizer has the same po-larization direction as one of the images,only this image will pass the LCD.The other image is completely blocked by the LCD because of 90polarization difference.Hence,the image on LCD will be seen at the space where the image is converging.Next,the polarization switch changes the incoming light’s po-larization direction.Then the polarization direction of the other image is matched to that of the LCD panel polarizer.The LCD panel image is now seen at the other side.Hence,it is possible to display full panel resolution stereoscopic images based on time multiplexing.The latter is basically a stereoscopic images based on a spatial multiplexing.For full panel resolution,two displays are combined by a half mirror for to view the displays at a common direction.To direct each display to its corresponding eye,the polarization directions of two panels are 90different to each other and a polarized eyeglasses is needed.Instead of using the eyeglasses,the common direction is blocked by locating two polarizer pieces side by side.The polarization direction of each piece is the same as that of its corresponding display panel.Then these pieces are imaged by a concave mirror to form viewing zone and the images on the two panels are imaged by a lens to the mirror.The display part can be replaced by

a –pole LCD.IV .C HARACTERISTICS OF M ULTIVIEW 3D I MAGE AND

H OLOGRAPHIC I MAGE

In the physical optics including hologram,the object depth is transformed into phase.Hence to preserve the depth informa-tion in the hologram,phase should be recorded.In the hologram,phase is recorded as interference patterns formed by interfering two coherent beams.i.e.,reference and object beams [39].For the two objects of different distances,the phase difference be-tween the rays from the longer distance object and the refer-ence beam will be smaller than those for the other object.This is because the beam incidence angle to the photoplate for the near distance object is bigger than for the far distance object.Hence hologram can reconstruct original object image with the real depth.Hologram is also providing continuous parallax [40].The continuous parallax means that the parallax value of an ob-ject in the hologram is not a discrete but continuously changing when viewers move their eyes from a point to another point in the hologram as viewers see things in their living environments.This is possible in the hologram because the object information is recorded on the entire hologram surface.

In contrast to the hologram,the multiview images comprise no phase information and continuous parallax.The difference between taking a hologram and a photo is depicted in Fig.6.For the case of the hologram,a photoplate located at the en-trance pupil of the camera objective records re?ected rays from the various surface of the object to the whole pupil plane by in-terfering the rays with a reference beams.The reference and illu-minating beam to the object should be originated from a visible laser.Since every point on the object re?ects rays to all over the pupil plane,continuous parallax and object phase information are preserved.When two objects with different depth are used to record a hologram,the re?ected rays from two objects will be overlapped at each point of the pupil plane.However,these rays keep their identities because their different phases relative to the reference beam allow them to be recorded independently rather than mixing.

For the case of the photo,the information on the pupil is trans-formed into a 3D object image by the objective and recorded on the photo ?lm as a plane image.The plane image has a pressed form of the 3D image because all rays from the 3D object image are passing through the ?lm.Due to the processing,much of the 3D image data are lost by the pressing and each point on the object may not ?nd a corresponding point on the ?lm.Since the transform of 3D to 2-D reduces surface area of the 3D image,some object surfaces are deformed and consequently some sur-face points are overlapped to each other in the plane image.These overlapped points will form a new point which brightness is equal to the sum of the brightness of all overlapped points,

6JOURNAL OF DISPLAY

TECHNOLOGY

https://www.360docs.net/doc/403250948.html,parison of IP and reconstructed image of a hologram.

because no phase is involved.Hence original object informa-tion is lost.Furthermore,when two objects with the same size but different distances are imaged,their images are recorded in the same?lm plane with a different sizes and positions.Hence the existence of two objects is identi?ed but the size difference can hardly tell that which object is near the camera unless there is at least one visual clue in the recorded image.The parallax and phase information are not there in the camera image.This is why at least two cameras are required to identify which ob-ject is nearer.Two cameras in side by side can provide parallax information because the relative position of the object image in two camera?lm planes are different to each other.However, this parallax has a?xed value which depends on the distance between two cameras.Aligning more cameras side by side can increase the number of the discrete parallax values and reducing between camera distances will make the parallax values smaller but continuous parallax can be hardly achieved with a multiview camera arrays.The situation will not be different for IP(Inte-gral Photography)[41].IP is a kind of multiview image methods [42].However,it uses the multiview images obtained through its VZFO.The VZFO is a microlens array/similar kinds and each microlens in the array works as a camera objective to obtain the images.The image obtained with each microlens is named as elemental image.Hence the number of the multiview images, i.e.,elemental images in the IP is the same as the number of microlenses in the microlens array.Fig.7(a)shows the image forming principle in IP.Two objects speci?ed as1and2are located at a point within the depth of focus region and outside of the region,respectively.These two objects are imaged on an image sensor by each microlens according to general lens image forming principle.The image position in the image sensor for each microlens is de?ned as the crossing point of the ray from the object to the center of the microlens,i.e.,chief ray and the sensor.After recording,left and right sides of each elemental image is swapped for image inversion(For the following de-scriptions,this process is omitted for the simplicity)and then displayed on a display panel which has the same pixel size and resolution as the image sensor with the same VZFO.Since the elemental images are on the same display panel,they are im-aged at the same focused plane by their front microlenses.The focused image positions of the object1and2images from each microlens on the focused plane are de?ned as the cross points of the chief rays and the focused plane.As shown in Fig.7(a), object1images from microlenses are almost matched because it was very close to the focused plane but for object2,the focused images from each microlens is completely separated from each other.However,all focused images are diverging with the angle equal to the converging angle as shown by the object2images. These defocused images are completely matched to each other at the original object2position and form a matched defocused image.For the case of the object1,the same defocused images are matched at the original object1position,as the object2,but the matched defocused image is not seen due to its small size. The presence of the matched defocused images will also exist in the typical multiview3D image displays if the image plane is not in the focal plane of VZFO.These matched defocused im-ages are confusingly regarded as a focused image because they appear at the object distances,are bright and clear when they are within the range of depth of focus.It seems that the presence of these matched defocused images has given such a illusion that IP can display a hologram like image[43].However,these matched defocused images are different from the hologram image be-cause these images are defocused plane images having only spa-tial position.No object depth is preserved.Furthermore,each of the matched defocused images is seen only at near the original object position.This will be the clear indication of no contin-uous parallax in IP,if the depth sense in IP is only given by these images.If the defocused images are dominating factors of forming the3D images in the IP displays,IP will not be a good3D display method because:1)the perceived images will be severely distorted due to the defocused image size increase as the object distance from the focused image plane increases

SON et al.:RECENT DEVELOPMENTS IN 3-D IMAGING TECHNOLOGIES

7

Fig.8.Ideal ray propagating geometry of the focused point array.

and 2)only part of the IP displays can be viewed to viewers due to the dif?culty of seeing all the defocused images at the same time.However,these phenomena are not visually perceived in the real IP displays.This means that the dominating factors are not the defocused images but parallaxes as in other multiview 3D imaging methods.Fig.7(b)shows the reconstruction geom-etry of a hologram.When eyes are moving across the hologram,each eye sees two object images with different depths simulta-neously.Since,the positions of each object are ?xed without regard to the moving,the moving allows seeing different sides of the objects without interruption.This will generate a con-tinue parallax.The continuous parallax is like the parallax that a viewer can achieve when he/she sees outside from inside of a moving car.A possible way of the continuous parallax with the multiview imaging methods is minimizing disparities between adjacent images in a multiview image set and directing many different view images within viewers’eye pupil.If this contin-uous parallax alone can impart the real depth information,real depth information will also be obtained with multiview image methods.This is the basic idea for the image and pixel array methods mentioned in Section III.These methods are trying to directing many different view images with a small direction angle difference between them.

V .P ROGRESSES IN THE I MAGE AND P IXEL A RRAY M ETHODS To direct different view images to different directions,the same position pixels

from different view images with hor-izontal parallax are directed to different horizontal directions with an equal angle distance and then angularly multiplexed to be combined as a focused point

of pixels [25].If each view

image consists of

horizontal

vertical pixels,the focused point array also consists

of

focused points.These focused points are scanned in both horizontal and vertical directions on a diffusing screen to display full frames of

the different view images.Then individual pixels in the point will diverge to the direction which they hold before combining.This is possible because the screen diffuses only vertically.The vertical diffuser screen is used to direct pixels in different heights on the screen to viewers’eyes.The pixels belong to a view image are propa-gating in parallel to each other in the horizontal direction.The rays from the pixels are almost collimated,i.e.,the images are collimated.If the angle distance between adjacent pixels in the point is set to not less than the diverging angle of each pixel,the neighboring images will not be mixed.

Hence different view images are

propagating different horizontal directions.How-ever,the opposite case,i.e.,when the angle distance is smaller than the diverging angle,neighboring view images are mixed and the images look blurred.In this case,it will be better to view the screen at a short distance.In this method,since each view image is collimated,viewers can not see the whole image,viewers can see only a part of images within their eyes’pupil sizes.To make a bigger part of an image to be seen,the fo-cused point distance should be smaller.However,the part of an image is not an actual image projected to viewers’eyes but a combined image with a pixel from each image

of images as shown in Fig.8.Fig.8shows the ideal ray propagating geom-etry of the focused point array.There are ?ve focused points and each focused point emits 41different direction rays.Since each ray represents a pixel and the pixel is actually expanded,the gap between two rays can be considered as the pixel width.In Fig.8,

there are forty one different view images,

i.e.,

,and each view image is composed of ?ve pixels.The viewing zone where all these ?ve focused points are seen simultaneously is de?ned as the ray crossing region of two end points.It is marked by thick lines.Within this region,viewers see a pixel from each point from all ?ve points at any place in the region.The image com-positions at all areas surrounded by the rays are different from each other.Hence huge number of new images will be perceived

in the viewing zone.The number can be proportional

to

,

where

and are the number of rays from each point,within the viewing zone and total number of focused points,respec-tively.The clear diamond shaped regions are the place where viewers can have a room to shift eyes without seeing different images.If the diamond widths are smaller than the pupil size of

8JOURNAL OF DISPLAY TECHNOLOGY

viewers’eyes,two different images can simultaneously get into each eye.Can this generate continuous parallax?It is also uncer-tain that these tremendous numbers of new images can impart continuous parallax.If the widths of diamond become longer than viewers’interocular distance,two eyes can see the same image.Since the diamond widths are increasing with the dis-tance from the focused point array,the possible viewing distance range can be de?ned by the width of the diamond.The broken lines de?ne the positions where diamonds with the same sizes are created.All these images are combined images of a pixel from each point from all points.The existence of these com-bined images is veri?ed experimentally in the point light source array method[44].It is noticed that the visual image quality of the combined images are somewhat worse than individual view images[45].But the methods are somewhat different to each other,the image quality of the pixel array method should be ex-amined carefully in relation with the continuous parallax for fur-ther developments.

Instead of scanning the focused points,many collimated images of different horizontal directions can be projected to the diffuser plane simultaneously[26].This method is a spatial multiplexing version of the focused point array.The focused point array is based on a spatio-temporal multiplexing method to display images.There can be other versions of the focused point array.One of them will be replacing the vertical line of the focused point array with an image point array as shown in Fig.9.The image from each projector in the array is collimated and then focused to a point to replace each point of the focus point array.As the projector,a type of pico-projectors can be used.The image from each projector is composed of a vertical line of the focused point array.The diffusing screen can be placed to any location within the viewing zone in parallel to the broken lines in Fig.8.The screen size will be the governing factor of determining the location.Since the projected image is expanding in both directions,the diffusing power of the screen in vertical direction should be more than the focused point array case.

VI.Q UALITIES OF3D I MAGES

As described in previous sections,new3D imaging methods are continuously developed but the difference is only in the mechanisms for directing images to viewers’eyes.As the new forms of displays,viewing3D images of an object from all sides of a cube and a table type3D displays display based on many projectors aligned along a circle are also introduced [46].Since3D displays have a limited viewing space and are application oriented,more mechanisms will be developed in future.However,the underlying principles of depth perceiving will not be different.Hence measuring qualities of3D images generated by different3D imaging methods is only way of dis-criminating one method with other methods.Of course the ideal 3D image is the one which can deliver the same image quality and viewing conditions as we experience from our surrounding environment.But current3D displays are far from to display the ideal one.Hence identifying and quantifying factors causing the differences and setting standard values are utmost important for reducing the differences.Factors affecting the3D image quality are coming from two sides,i.e.,human and display

[47].Fig.9.A display based on an image point array.

Since human factors are related with human’s psychophysical conditions,they cannot be easily identi?ed.However,regarding the display,the factors can be easily identi?ed because they are closely related with design parameters of the display and can be quanti?ed by measurements.Some of these factors are viewing distance,viewing zone angle and viewing zone depth, differences between different view images such as disparity, brightness,resolution and color,perceived depth range,depth resolution,crosstalk,moire,number of different view images, and kind and direction of viewing zone forming optics.These factors are mostly related with multiview3D imaging displays. Hence disparity,crosstalk,moiré,and number of different view images are not suit for electro and digital-holography[20],[21], [48],[49]and volumetric imaging methods,though the number of different view images can be replaced with the number of layers in the layered volumetric images.For the multiview 3D images,crosstalk is the most frequently measured factor and moirés are visually the most annoying quantity so far. The crosstalk gives also brightness difference information for different view images because it measures the intensity dis-tribution across the viewing zone cross section.To reduce the crosstalk,pixel structure and shape were modi?ed to?t better to the employed VZFO.These modi?cations results not only reduced the crosstalk but at the same time increased brightness too[17].Moirés in the3D displays is not even caused by the regularity between VZFO and panel’s pixel structure but also between VZFO and prism sheets in the back light panel of the LCD[50].Regarding the former,the theoretical solutions for minimizing moirés for display panels with different pixel aspect ratios had been derived and its effectiveness had been proved[51],[52].The solution is adjusting the crossing angle of VZFO and the display panel.But applying a diffusive adhe-sive between VZFO and the panel is also effective[53].Both of these methods are effective for the latter,but with diffuser. For the measuring the volume of viewing zone,measuring interocular luminance difference is suggested[14],[15].

SON et al.:RECENT DEVELOPMENTS IN3-D IMAGING TECHNOLOGIES9

VII.C ONCLUSION

Even in the3D displays,the3D image quality is preferred more than the viewer comfort and the natural depth.This is why stereoscopic imaging methods with full panel resolution for each component image is more popular in commercial sector than multiview3D images.To get concerns from the commer-cial sector,the multiview3D images may need to wait until a very high resolution display panel for multiview image display and VZFO which will cause no deterioration in image quality,a new multiview imaging method which can relieve the require-ments,and/or components which prices doesn’t cause the3D display costs much more than the plane image,are developed. It is expected that the quality requirements for future3D dis-plays will be higher than now.Hence the guidelines for mea-suring the quality of3D images and standard methods of mea-suring factors affecting the quality should be developed in near future.Furthermore,more efforts should be given to develop new?at panel displays,and new methods based on holography and super-multiview,and to modify the characteristics of ex-isting display panels for3D displays with better quality.Among the qualities,depth resolution of3D imaging will become a major factor for determining the quality because users can in-teract with3D scenes on the displays in near future.The inter-action accuracy will be governed by the resolution.

R EFERENCES

[1]J.-Y.Son and B.Javidi,“3-dimensional imaging systems based on mul-

tiview images,”J.Display Technol.,vol.1,no.1,pp.125–140,Sep.

2005.

[2]J.-Y.Son,B.Javidi,and K.-D.Kwack,“Methods for displaying3di-

mensional images,”Proc.IEEE(Special Issue on:3-D Technologies

for Imaging&Display),vol.94,no.3,pp.502–523,Mar.2006.

[3]B.Javidi,F.Okano,and J.-Y.Son,Three Dimensional Imaging,Visu-

alization,and Display.New York:Springer,2009.

[4]A.Stern and B.Javidi,“3D image sensing,visualization,and pro-

cessing using integral imaging,”Proc.IEEE,vol.94,no.3,pp.

591–608,Mar.2006.

[5]R.Martinez-Cuenca,G.Saavedra,M.Martinez-Corral,and B.Javidi,

“Progress in3-D multirespective display by integral imaging,”Proc.

IEEE,vol.97,no.6,pp.1067–1077,Jun.2009.

[6]R.Martinez-Cuenca,G.Saavedra,M.Martinez-Corral,and B.Javidi,

““Extended depth-of-?eld3-D display and visualization by combi-

nation of amplitude-modulated microlenses and deconvolution tools,”

Progress in3-D multirespective display by integral imaging,”J.Dis-

play Technol.,vol.1,no.2,pp.321–327,oct.2005.

[7]H.Nam,B.-S.Kim,C.Y.Park,J.Gu,and H.Chung,“3D display in

mobile applications,”in SID Proc.6th Int.Meeting on Inf.Display,

Exco,Korea,2006,pp.1561–1564.

[8]D.Suzuki,T.Fukmi,E.Higano,N.Kubota,T.Higano,S.Kawaguchi,

Y.Nishimoto,K.Nishiyama,and K.Nakao,“Crosstalk-free3D display

with time-sequential OCB LCD,”in SID’09Dig.,2009,pp.428–431.

[9]J.Schultz,R.Brott,M.Sykora,W.Bryan,T.Fukami,K.Nakao,and

A.Takimoto,“Full resolution autostereoscopic3D display for mobile

applications,”in SID’09Dig.,2009,pp.127–130.

[10]S.Kim,B.You,H.Choi,B.Berkeley,D.Kim,and N.Kim,“World’s

?rst249Hz TFT-LCD technology for full-HD LCD-TV and its appli-

cation to3D display,”in SID’09Dig.,2009,pp.424–438.

[11]S.-J.Park,M.-C.Park,and J.-Y.Son,“Stereoscopic imaging and dis-

playing for3-D mobile phones,”Appl.Opt.,vol.48,pp.H238–H243,

2009.

[12]T.Balogh,“The HoloVizio system,”SPIE,vol.6055,pp.60550U-

1–12,2006.

[13]J.-Y.Son,“3dimensional image device based on focused image array,”

Korea Patent,Feb.5,2009.

[14]G.Hamagishi,K.Taira,K.Izumi,S.Uehara,T.Nomura,K.Mashitani,

A.Miyazawa,T.Koike,A.Yuuki,T.Horikoshi,Y.Yoshihara,Y.Hisa-

take,H.Ujike,and Y.Nakano,“Ergonomics for3D displays and their

standardization,”in IDW’08,Proc.5th Int.Display Workshops,2008,

pp.1099–1102.[15]K.Taira,G.Hamagishi,K.Izumi,S.Uehara,T.Nomura,K.Mashitani,

A.Miyazawa,T.Koike,A.Yuuki,T.Horikoshi,Y.Hisatake,H.Ujike,

and Y.Nakano,“Variation of autostereoscopic displays and their mea-surement,”in IDW’08,Proc.15th Int.Display Workshops,2008,pp.

1103–1106.

[16]A.Yuuki,S.Uehara,K.Taira,G.Hamagishi,K.Izumi,T.Nomura,

K.Mashitani,A.Miyazawa,T.Koike,T.Horikoshi,and H.Ujike,“Viewing zones of autostereoscopic displays and their measurement methods,”in IDW’08,Proc.15th Int.Display Workshops,2008,pp.

1111–1114.

[17]D.-S.Kim,S.Shestak,K.-H.Cha,J.-P.Koo,and S.-D.Hwang,“De-

velopment of time-multiplexed autostereoscopic display based on LCD panel,”in Three-Dimensional Imaging,Visualization,and Display,B.

Javidi,F.Okano,and J.-Y.Son,Eds.New York:Springer,2009,pp.

467–491.

[18]C.Chen,Y.Huang,S.Chuang,C.Wu,H.Shieh,W.Mphepo,C.Hsieh,

and S.Hsu,“Liquid crystal panel for high ef?ciency barrier type au-tostereoscopic three-dimensional displays,”Appl.Opt.,vol.48,no.18, pp.3446–3454,2009.

[19]F.Yaras,H.Kang,and L.Onural,“Real-time phase-only color holo-

graphic video display system using LED illumination,”Appl.Opt.,vol.

48,no.34,pp.H48–H53,2009.

[20]B.Munjuluri,M.Huebschman,and H.Garner,“Rapid hologram up-

dates for real-time volumetric information displays,”Appl.Opt.,vol.

44,no.24,pp.5076–5085,2005.

[21]T.Shimobaba,A.Shiraki,N.Masuda,and T.Ito,“Electroholographic

display unit for three-dimensional display by use of special-purpose computational chip for holography and re?ective LCD panel,”Opt.Ex-press,vol.13,no.11,pp.4196–4201,2005.

[22]T.Senoh,K.Yamamoto,R.Oi,T.Mishina,and M.Okui,“Computer

generated electronic holography of natural scene from2D multi-view images and depth map,”in20082nd Int.Symp.on Universal Commun., 2008,pp.126–133.

[23]N.T.Shaked,J.Rosen,and A.Stern,“Integral photography:

White-light single-shot hologram acquisition,”Opt.Express,vol.15, pp.5754–5760,2007.

[24]J.-Y.Son,V.V.Saveljev,Y.-J.Choi,J.-E.Bahn,and H.-H.Choi,

“Parameters for designing autostereoscopic imaging systems based on lenticular,parallax barrier and IP plates,”Opt.Eng.,vol.42,no.11,pp.

3326–3333,2003.

[25]Y.Kajiki,H.Yoshikawa,and T.Honda,“Ocular accommodation by

super multi-view stereogram and45-view stereoscopic display,”in IDW’96,Proc.11th Int.Display Workshops,1996,pp.489–492. [26]Y.Takaki and H.Nakanuma,“Improvement of multiple imaging

system used for natural3D display which generates high-density irectional images,”in Proc.SPIE,2003,vol.V5243,pp.43–49. [27]S.Wakita,Y.Sumitomo,Y.Kobayashi,H.Yamamoto,and S.Suyama,

“High-speed liquid-crystal optical de?ector for3-D display,”in SPIE, 2009,vol.7237,pp.723724-1–7.

[28]Y.Kao,Y.Huang,K.Yang,P.C.-P.Chao,C.Tsai,and C.Mo,“An

auto-stereoscopic3D display using tunable liquid crystal lens array that mimics effects of GRIN lenticular lens array,”in SID’09Dig.,2009,pp.

111–114.

[29]J.Yan,S.Kowel,H.Cho,C.Ahn,G.Nordin,and J.Kulick,“Au-

tostereoscopic three-dimensional display based on a micromirror array,”Appl.Opt.,vol.43,no.18,pp.3686–3696,2004.

[30]Nakai,K.Hoshino,K.Matsumoto,and I.Shimoyama,“Multiview au-

tostereoscopic display with double-sided re?ecting scanning micromir-rors,”SPIE,vol.6055,pp.60551F-1–9,2006.

[31]U.V ogel,L.Kroker,K.Seidl,J.Knobbe,C.Grillberger,J.Amelung,

and M.Scholles,“OLED backlight for autostereoscopic displays,”

SPIE,vol.7237,pp.72370U-1–9,2009.

[32]M.A.Klug,C.Newawanger,Q.Huang,M.e.Holzbach,and,

Zebra Imaging Inc.,“Active digital hologram displays,”U.S.Patent 7227674,Jun.5,2007.

[33]D.Pasltis and F.Mok,“Holographic memories,”Scienti?c Amer.,vol.

273,no.5,pp.70–76,1995.

[34]P.Blanche,S.Tay,R.V oorakaranam,P.Saint-Hilaire,C.Christenson,

T.Gu,W.Lin,D.Flores,P.Wang,M.Yamamoto,J.Thomas,R.Nor-wood,and N.Peyghambarian,“An updatable holographic display for 3D visualization,”J.Display Technol.,vol.4,no.4,pp.424–430,Dec.

2008.

[35]C.Chen,Y.Yeh,and H.Shieh,“3-D mobile display based on moire-

free dual directional backlight and driving scheme for image crosstalk reduction,”J.Display Technol.,vol.4,no.1,pp.92–96,Mar.2007.

[36]J.C.Schultz,R.Brott,M.Sykora,W.Bryan,T.Fukami,K.Nakao,and

A.Takimoto,“Full resolution autostereoscopic3D display for mobile

application,”in SID’09Dig.,2009,pp.127–130.

10JOURNAL OF DISPLAY TECHNOLOGY

[37]D.M.Hoffman,P.J.W.Hands,A.K.Kirby,G.D.Love,and M.S.

Bank,“Stereo display with time-multiplexed focal adjustment,”SPIE,

vol.7237,pp.72370R-1–12,2009.

[38]T.Honda,Y.Kuboshima,K.Iwane,and T.Shiina,“Development of

auto-stereoscopic display system,”SPIE,vol.6055,pp.60550V1–8,

2006.

[39]R.J.Collier,C.B.Burckhardt,and L.H.Lin,Opt.Holography,Student

ed.New York:Academic,1971,ch.1,pp.15–21.

[40]M.C.Forman,N.Davies,and M.McCormick,“Continuous parallax

in discrete pixelated integral three-dimensional displays,”J.Opt.Soc.

Amer.,vol.V20,no.3,pp.411–420,2003.

[41]J.-S.Jang and B.Javidi,“Improved viewing resolution of three-dimen-

sional integral imaging with nonstationsary micro-optics,”Opt.Lett,

vol.V28,pp.324–326,2003.

[42]J.-Y.Son,S.-H.Kim,D.-S.Kim,and B.Javidi,“Image forming prin-

ciple of integral photography,”J.Display Technol.,vol.V4,no.3,pp.

324–331,Sep.2008.

[43]F.Okano,J.Arai,K.Mitanim,and M.Okui,“Three-dimensional in-

tegrated television using high-resolution video system with2000scan-

ning lines,”in Three-Dimensional Imaging,Visualization,and Display,

B.Javidi,F.Okano,and J.-Y.Son,Eds.New York:Springer,2009,

pp.3–23.

[44]V.V.Saveljev,J.-Y.Son,S.-H.Kim,and D.-S.Kim,“Image mixing

in multiview3-D imaging systems,”J.Display Technol.,vol.V4,no.

3,pp.319–323,2008.

[45]J.-Y.Son,V.V.Saveljev,J.-S.Kim,S.-S.Kim,and B.Javidi,“Viewing

zones in3-D imaging systems based on lenticular,parallax barrier and

microlens array plates,”Appl.Opt.,vol.43,no.26,pp.4985–4992,

2004.

[46]S.Yano,“Considerations about the development of three-dimensional

display systems adopted for human visual?eld,”in IMID2009Dig.,

2009,pp.1183–1186.

[47]T.Izumi,Fundamental of3-D Imaging Techniques,Japanese ed.

Tokyo:NHK Science and Technology Lab.,Ohmsa,1995,ch.3,pp.

37–69.

[48]Y.Frauel,T.Naughton,O.Matoba,E.Tahajuerce,and B.Javidi,

“Three dimensiaonl iamging and display using computational HOLO-

GRAPHIC imaging,”Proc.EEE,vol.94,no.3,pp.636–654,Mar.

2006.

[49]P.Ferraro,S.Grilli,L.Al?eri,S.De Nicola,A.Finizio,and B.Javidi,

“Full color3D imaging by digital holography and removal of chromatic

aberrations,”J.Display Technol.,vol.4,no.1,pp.97–100,Mar.2008.

[50]S.Uehara,T.Hiroya,K.Shigemura,and H.Asada,“Reduction and

measurement of3D moire caused by lenticular sheet and backlight,”in

SID’9Dig.,2009,pp.432–435.

[51]Y.-C.Song,V.V.Saveljev,J.-Y.Son,S.-W.Yeom,and Y.A.

Vashpanov,“Conditions for moire free contact-type3dimensional

displays,”J.Opt.Soc.Korea,vol.V12,no.2,2008.

[52]V.V.Saveljev,J.-Y.Son,and K.-H.Cha,“About a moire-less con-

dition in non-square grids,”J.Display Technol.,vol.V4,no.3,pp.

332–339,2008.

[53]J.-T.Lien,M.-D.Chou,F.-C.Lu,C.-C.Chang,C.-L.Tsou,T.-Y.Lin,

and C.-N.Mo,“Using diffusive adhesives to reduce moirépattern of

autostereoscopic displays,”in IDW’08,Proc.15th International Dis-

play Workshops,2008,pp.

2083–2086.

Jung-Young Son received the Ph.D.degree in engi-neering science(majoring in optics)from University of Tennessee,in1985.

He is currently a Chair Professor at School of Computer and Communication Engineering,Daegu University,Kyungsan,Kyungbuk,Korea.His re-search interests include three-dimensional imaging, electro-holography and optical measurements and instrumentations.He has more than300papers,60 registered patents,and5books to his credit.

Dr.Son is a SPIE

fellow.

Bahram Javidi(S’82–M’83–SM’96–F’98)received

the B.S.degree from George Washington University,

Washington,DC,and the M.S.and Ph.D.degrees

from the Pennsylvania State University,University

Park,all in electrical engineering.

He is the Board of Trustees Distinguished Pro-

fessor at the University of Connecticut which is

the highest rank and honor bestowed on a faculty

member based on research,teaching,and service.

He has over630publications.He has completed8

books and44book chapters.He has published over 250technical articles in major peer reviewed journals.He has published over 330conference proceedings,including over110Plenary Addresses,Keynote Addresses,and invited conference papers.His papers have been cited over 5500times according to the citation index of WEB of Science.

Dr.Javidi is Fellow of seven scienti?c societies,including Institute of Elec-trical and Electronics Engineers(IEEE),American Institute for Medical and Biological Engineering,Optical Society of America,and Institute of Physics. In2008,he received a Fellow award by John Simon Guggenheim Foundation. In2010,he was the recipient of George Washington University’s Distinguished Alumni Scholar Award,University’s highest honor for its alumni in all disci-plines.He received the2008IEEE Donald G.Fink prized paper award among all (over180)IEEE Transactions/Journals,and Magazines.n2007,The Alexander von Humboldt Foundation awarded him the Humboldt Prize for outstanding US scholars.In2005,he received the Dennis Gabor Award in Diffractive Wave Technologies from the International Society for Optical Engineering(SPIE). Early in his career,the National Science Foundation named him a Presidential Young Investigator,and he received The Engineering Foundation and the Insti-tute of Electrical and Electronics Engineers Faculty Initiation Award.He was selected in2003as one of the nation’s top160engineers between the ages of 30–45by the National Academy of Engineering to be an invited speaker at The Frontiers of Engineering

Conference.

Sumio Yano received the B.A.and Ph.D degrees in

engineering from the University of Electro-Commu-

nications in1977and1993,respectively.

He joined NHK in1977,and started to work at

NHK Nara Broadcasting Station.He was transferred

to NHK Science and Technical Research Laborato-

ries in1981.He was a loan employee at ATR Human

Information Processing Research Laboratories from

1992to1995and ATR Human Information Science

Research Laboratories from2002to2004,respec-

tively.Currently,he is on loan to National Institute of Information and Communications Technology from2006.He has studied about the human visual system and its application of image

systems.

Kyu-Hwan Choi received the B.E degree in physics

from Kyungwon University,M.E and Ph.D.degrees

in physics from Korea University,South Korea,in

2001,2003,and2009,respectively.

His main research?eld of study was3D display

system,digital holography,and multi-view display

system.He is currently a R&D Staff Member

at Samsung Electronics Company,Ltd.,Yongin,

Gyeonggi,Korea.