Home

There are several methods of camera calibration. Of these methods, the goniometer method makes use of a very precise grid plate placed on the image plane of a camera and the angles responding to the grid intersection from an object are measured through the goniometer, which in turn are compared to nominal and real values to find the distortion values. In the collimator method, a collimator set-up with many defined yet different angles is used to project the test patterns on an image plane. The camera focus is set to infinity and the measured nominal values and real values are compared to find the parameters of interior orientation. Compared to such methods, a simpler and generally used method is the camera calibration model studied by Zhang [5] and Tsai [13], which applies the image forming principles of pinhole cameras. In this model, a three-dimensional object point is projection a two-dimensional image plane of the camera.

Let the three-dimensional object points as P_o = (X_o,Y_o,Z_o) in the world frame. And their projection on the two-dimensional image plane of the camera as P_c = (X_c,Y_c,Z_c) in the camera frame. Between the P_o and P_c are related to the location of the image points geometrically rotates and moves translationally. Because of this, the translational vector T and rotation matrix R can be used to represent such changes using the rigid body motion equation P_c = R +P_o +T. The relation between the image points P_c = (X_c,Y_c,Z_c) of the camera sensor plane and the object points P_o = (X_o,Y_o,Z_o), can be represented as[5][13]

where (x_i, y_i) are the normalized coordinates of P in the image sensor frame, f is the effective focal length. However, actual cameras in use do not form images like a pinhole camera using the rectilinearly of light rays but rather forms images using a lens system, which results in the representation of lens-induced aberrations on the image. In particular, distortions between the plane- based camera image recording sensors and the image forming properties between the object and the lens, are the main reason behind changes of an entire image. Distortions are classified into radial distortions that present radial displacements of image points on the image plane and tangential distortions, a type of de-centering distortion, that results when the center of curves of lens surfaces of the lens system are not accurately on top of the same optical axis. An approximation of the coordinate system based on radial distortion can be represented as [4].

where k₁, k₂, k₃ refer to the radial distortion coefficients and r² = x² + y². Tangential distortion can be represented as

where k₄ and k₅ refer to the tangential distortion coefficients. For simple use, the coordinate system of the distorted image points including both radial and tangential distortions was represented as (x_d, y_d) and in consideration that images form on camera sensor device surfaces in pixel units, the pixel coordinates [ u,v ] within the sensor surface were rewritten as

where c_x and c_y refer to the coordinates of the principle points, α refers to the aspect ratio, and a f_u and f_v refer to the focal lengths in pixel units found by dividing the lens focal length by the size of the image sensor. Thus, this model includes internal parameters such as f, α, x_o and y_o distortion coefficients k₁, k₂, k₃, k₄ and k₅ as its parameters. The goal of the camera calibration process is the determination of the optimal values of such parameters based on the known two-dimensional or three-dimensional observed target image.

Chessboard patterns are composed of small black and white squares and the corner points of each square become the characteristic points. The corner points of the squares are detected using the Harris corner detection method [16].

where [M (x,y)] refers to local autocorrelation function related matrices and det[M ] and trace [M ] refer to the trace and determinant values of each matrix [M ], respectively. Should the R (x,y) value of a point exceeds the predetermined limit value, the point is perceived as a corner. In general, precision of the Harris detection method is at the pixel level.

Optical noise of images refers to the random distribution of light according to the frequency and intensity added to signal intensity. At this point, should the noise present within the rectangular grid-type sample, given from within the image, have a normal distribution with a zero average, such noise is considered to be Gaussian noise or white noise. In addition, speckle noise refers to a type of optical image noise that multiplies equally distributed random noise having an average value of zero that randomly changes in intensity with an optical image having an intensity distribution of I [17][18].

A two-dimensional chessboard pattern was used as the reference coordinate system that formed the image. The used black and white chessboard pattern was square-shaped having a side length of approximately 15.9 mm and 10 × 14 sequences were printed and used. The Microsoft HD5000, used for the experiment, had a CMOS sensor still image pixel resolution of 1280 × 800, an optical lens system having a diagonal field of view of 66^o, and a device that automatically adjusts exposure and focus. The chessboard pattern was used to capture various images at a distance of 45 cm ∼ 80 cm at various angle and slope rotations and tilts as well as left and right movements. Also, for each condition, approximately 11 ∼ 13 different angles of photos were used. Image capturing was undertaken using a computer, and Fringe Processor (Bias Co.) software and camera calibration was undertaken using the camera calibration toolbox [19] developed by Jean-Yves Bouguet at Matlab (Mathworks company). For the purpose of measuring the effects that the image noise added to the chessboard pattern target, which is the reference of the coordinate points, had on camera calibrations, several measurements of the optical images added with Gaussian noise and Speckle noise. The noise of each levels having zero average values according to noise intensity at different noise distribution changes were taken. Fig. 1 presents a schematic diagram of the measurement system.

Fig. 1. 
Camera calibration measurement system

Fig. 2 presents the chessboard patterns applied with different intensity levels of Gaussian noise and Speckle noise. The added noise levels between levels of 0.001 and 0.25 were categorized into 7 levels.

Fig. 2. 
Chessboard patterns added with the noise used for calibration, (a) Gaussian noise level: 0.001, (b) Speckle noise level: 0.001, (c) Gaussian noise level: 0.1, (d) Speckle noise level: 0.1

Table 1 presents the standard deviation values of each noise intensity level for each of the noises.

Fig. 3 presents the locations of the chessboard patterns during the camera calibration process and the camera capturing process according to different rotating angles. The camera and captured chessboard pattern locations were changed as shown in the figure. However, increased noise levels resulted in cases in which the corner points of the squares of the chess patterns were not perceived. Because of this, chessboard patterns having different locations and rotation displacements were used in the calibration.

Fig. 3. 
Camera and chessboard pattern arrangement diagram to capture the image

Fig. 4 compares the standard deviation values of the Gaussian noise and speckle noise with different intensity levels and distributions as the data in Table 1. As shown in the figure, since the standard deviation of the two noise distributions is similar, the standard deviation value is used for the comparison of the experimental data.

Fig. 4. 
Relationship between the noise intensity level and the standard deviation values of each of the two types of noise

Fig. 5, presents the reprojection error values within the plane of the characteristic points of the parameters found through the camera calibration process based on chessboard pattern images added with different intensity levels of two types of noises. Through this, the effects of Gaussian noise and Speckle noise on reprojection coordinates according to increases in noise intensity levels were observed.

Fig. 5. 
Gaussian noise and Speckle noise intensity level induced changes of reprojection error values, (a) Gaussian noise level: 0.001, (b) Speckle noise level: 0.001, (c) Gaussian noise level: 0.1, (d) Speckle noise level: 0.1

During the process of camera calibration, intrinsic parameters such as focal length, principle point, and skew as well as extrinsic parameters such as lens system-based distortion, rotation matrices, and translation vectors can be calculated. Fig. 6 presents changes in average values of focal length errors and principle point errors due to Gaussian noise and Speckle noise.

Fig. 6. 
Changes in focal length error and principle point error average values (pixel units) due to Gaussian noise and Speckle noise

The standard deviation values presented in Table 1 were used as the noise intensity level values during this process. As shown in the figure, in the case of Gaussian noise, when the noise intensity standard deviation increased by 7.93 times from 2.35 to 18.65, average focal point length error increased by approximately 2.76 from 3.35 pixels to 6.57 pixels and the ratio between focal length error increases and noise increases was found to be ΔFLE/ΔNoise = 2.76/7.93 = 0.35. The average principle point error was found to increase by 1.52 times from 1.86 pixels to 2.82 pixels and the ratio between principle point error increases and noise increases was found to be ΔMPLE/ΔMNSD = 1.52/7.93 = 0.19. In the case of Speckle noise, average focal length error was found to increase by 1.96 times from 3.35 pixels to 6.57 pixels when the noise intensity level standard deviation increased by 9.8 times from 2.11 to 20.7 and the ratio between focal length error increases and noise increases was found to be ΔFLE/ΔNoise = 1.96//9.8 = 0.2. Average principle point error was found to increase by approximately 1.79 times from 1.64 pixels to 2.94 pixels and the ratio between principle point error increases and noise increases was found to be ΔMPLE/ΔMNSD = 1.79/9.8 = 0.18. When examining the results regarding error increase ratios of focal lengths based on increases in intensity of noise, it was found that Gaussian noise resulted in 0.35/0.2 = 1.75 times greater error than Speckle noise. In the case of principle point error, Gaussian noise was found to result in 0.19/0.18 = 1.05 times greater error than Speckle noise.

Through lens distortions, the radial distortion values and tangential distortion values were found. Fig. 7 presents a comparison of the changes in the 3 coefficients of radial distortion k₁, k₂ and k₃ presented in equation (2). The results indicated that coefficient k₁ was found to have almost no change regarding increases in noise intensity levels of both types of image noises. In the case of the coefficient k₂ and k₃ Gaussian noise was found to have small effects in noise intensity level standard deviation values higher than 15 whereas Speckle noise was found to have a large effect at noise intensity level standard deviation values equal to or greater than 11.

Fig. 7. 
Comparison of the effects of Gaussian noise and Speckle noise on radial distortion coefficients

Fig. 8 presents a comparison of the changes in errors of the 3 coefficients of radial distortion, k₁, k₂ and k₃ , according to the two types of noises. In the case of Gaussian noise, the error of radial distortion coefficients did not present significant changes up to the noise intensity level standard deviation value of 18. However, in the case of Speckle noise, the coefficients were found to be largely affected at noise intensity level standard deviation values equal to or greater than 20.

Fig. 8. 
Comparison of the effects of Gaussian noise and Speckle noise on radial distortion coefficient errors

Fig. 9 and 10 presents the comparison between changes in error values and the two coefficients of tangential distortion, k₄ and k₅, presented in equation (3) according to the two types of noises. The results indicated that the values of coefficients k₄ and k₅ have a small value of 10^-3 pixels, and that k₄ has a negative value and is largely affected at noise intensity level standard deviation values of 10 and higher according to the two types of image noises. k₅ was found to be influenced at very small levels according to the different noise intensity levels of both Gaussian noise and Speckle noise. An examination of the changes in tangential distortion errors as presented in Fig. 10, indicated that the coefficient had a very small value of 10^-4 pixels yet changed according to noise increases.

Fig. 9. 
Comparison of the effects of Gaussian noise and Speckle noise on tangential distortion coefficients

Fig. 10. 
Comparison of the effects of Gaussian noise and Speckle noise on errors of tangential distortion coefficients

Fig. 11 presents the average reprojection error values found during the camera calibration process based on the addition of different intensity levels of Gaussian noise and Speckle noise to the image. The results indicated that Gaussian noise rather than Speckle noise had a greater effect under the same image noise intensity levels. In the case of Gaussian noise, mean reprojection error increased by approximately 1.31 times from 0.384 pixels to 0.504 pixels when the noise intensity level standard of deviation increased by 7.93 times from 2.35 to 18.65 and the ratio between mean reprojection error increases and noise intensity increases was found to be ΔMRPE/ΔNoise = 1.31/7.93 = 0.16.

Fig. 11. 
Comparison of the effects of Gaussian noise and Speckle noise on mean reprojection error

In the case of Speckle noise, when the noise intensity level standard deviation increased by approximately 9.8 times from 2.11 to 20.7, average reprojection error was found to increase by approximately 1.42 times from 0.344 pixel to 0.49 pixel, and the ratio between mean reprojection error increases and noise intensity increases was found to be ΔMRPE/ΔNoise = 1.42/9.8 = 0.14. A comparison of the mean reprojection error increase ratios found based on increases in Gaussian and Speckle noise, indicated that Gaussian noise-induced effects were 0.16/0.14 = 1.14 times greater than Speckle noise-induced effects.