Rolling Shutter Image Compensation Based on Humanoid Robot NAO

Figure 2. Effect of image distortion can be seen on both the round ball, and the rectangular goals. 3. 2-D Motion Compensation
Figure 3. Image of the fluorescent light three distinct light/dark cycles.
1. Introduction Cameras are the introduction central sensor of the robots in most RoboCup leagues. The vision system of humanoid NAO robot plays an important role in the overall perception system. Rolling shutters are commonly found in low-cost, low-power CMOS cameras. The NAO robot is equipped with this kind of cameras. Cameras that contain rolling shutters do not expose the entire image at one instance, as is done with a global shutter. Instead rolling shutters have pixels that have been exposed at different times and merged together to form a single image. The combination between new and old information causes distortions in the image when viewing an object with movement relative to the camera. So this paper covers central problem of this kind of sensor, the so-called rolling shutter in order to calibrate the image distortion caused by the NAO robot’s moving head. 2. Model of Rolling Shutter Cameras In a rolling-shutter camera each row of pixels is scanned in sequential order over time to create an image. An example of how a set of scanlines is exposed is shown in Figure 1. We assume that the exposure time within a scanline is instantaneous, i.e. that the peaks in Figure 1 have zero width but integrate to some non-zero constant. The effect of e being non-zero is motion blur within the scanline, but has no geometric effects. We can formalize a camera model by noting that each row corresponds to a different instant in time, assuming that the exposure is instantaneous.
Figure 1. Rolling-shutter cameras expose scanline sequentially; for some Firewire cameras the total exposure and the scanning rate may be inversely proportional to the framerate. Cameras that contain rolling shutters do not expose the entire image at one instance, as is done with a global shutter. The distortion will occur in the Nao robot in the RoboCup standard platform for it is installed with a camera that contains rolling shutter. The camera takes 30 images per second. The very reason is that NAO robot takes a picture with its head moving. These distortions can cause differences in the co-ordinates, as well as the shape of the objects that the robot has seen. Since the distortion stretches or compresses the image of objects, the co-ordinates derived from that image are also altered. The changed co-ordinates, in particular the tilt rate of an object, have the potential to cause problems when attempting to determine the location of the object. The distortion causes problems when the shape of the object is used for its identification or measurement. An example of which is the circle fitting on a ball image in the NAO robot. If the ball is no longer circular in shape, circle fitting loses some of its effectiveness. There are two main sources for this relative velocity that causes distortions in the image. The movement may be that of the object, or the movement of the camera itself. This paper just covers the image distortion caused by camera movement, which is the more contribution for the image distortion.
Rolling Shutter Image Compensation Based on Humanoid Robot NAO
Shuying Zhao1, Longwei Jiang1, Ying Liu1, Shancheng Jiang1, Haichao Feng1 and Weiqian Tao1
1. College of Information Science and Engineering, Northeast University, Shenyang 110004, China
E-mail: zhaoshuying@
Abstract. According to the requirement of RoboCup Standard Platform League (NAO) Rule Book, the author did some experiments to finish the League’s tasks based on the humanoid robot NAO. The image captured by a moving camera will have a distortion which is affected by the so called “rolling shutter”. The rolling shutter means that not all scanlines are exposed over the same time interval. This paper presents the model of rolling shutter of cameras and principle of capturing images. Then corrections to image distortion found on the NAO robots is described. At last the cases of reversing the effects of a rolling shutter on the images taken by the robot’s moving camera are presented. This approach improves the effectiveness of shape recognition and the accuracy of Tilt rate of field line
3.1. Delay for an Entire Image Before the image can be de-skewed, the timing of the camera should be known as is mentioned before. The magnitude of the delay determines the magnitude of the distortion on the image. The frame of the camera we used is 30fps, which is saying that the camera takes 30 pictures every second. But we don’t know the timing that camera spends on recording an image. To find this value an image was taken of a fluorescent light that flickers at a known frequency (100Hz). When viewed, the fluorescent light displayed three light and dark bands upon the image. This shows that the light went through three dark/light cycles in the time one image was taken. Given the period of the cycles (0.01s), the time taken to capture the frame was calculated to be approximately 0.03s (1). This value is very close to the frame rate of the camera, which operates at 30 frames per second. Therefore was assumed that the entire period between frames is used to capture the next frame. This gives an approximate period of 33.3ms. As which is shown in Figure 3.