Korean Institute of Information Technology

Current Issue

The Journal of Korean Institute of Information Technology - Vol. 20 , No. 11

[ Article ]
The Journal of Korean Institute of Information Technology - Vol. 20, No. 10, pp. 109-117
Abbreviation: Journal of KIIT
ISSN: 1598-8619 (Print) 2093-7571 (Online)
Print publication date 31 Oct 2022
Received 09 Aug 2022 Revised 30 Aug 2022 Accepted 02 Sep 2022
DOI: https://doi.org/10.14801/jkiit.2022.20.10.109

Fast Poissonian-Gaussian Noise Estimation
Hyun Sang Park*
*Professor, Div. of Electrical Electronic and Control Engineering, Kongju National University

Correspondence to : Hyun Sang Park Div. of Electrical Electronic and Control Engineering, Kongju National University, 1223-24, Cheonan-daero, Seobuk-gu, Cheonan-si, hungcheongnam-do, 331-717, Korea Tel.: +82-41-521-9157, Email: vandammm@kongju.ac.kr


Abstract

The noise is modeled as Poissonian-Gaussian as it is naturally suited for the raw-data of modern digital imaging sensors. The hard segmentation approach used in the conventional noise estimation schemes often degrades the accuracy of the estimation by reducing the sample size for estimation. Increasing the sample size, by using soft segmentation approach, improves the accuracy of the estimation. We presented a fast and practical noise level function estimation method for Poissonian-Gaussian noise from a single image. We initialize a pre-fixed number of intensity centroids based on the noisy image histogram, and compute the noise variances for each intensity centroid in the image intensity domain via weighted summation. The pixels, which have similar intensity with the intensity centroid and correspond to the low gradient region, are assigned with high weights and vice versa. Experiments on synthetic as well as real raw-images show that the proposed method provides reliable noise estimation 20 times faster than the state-of-the-art.

초록

이미지 센서의 잡음은 푸아송-가우스 방식으로 모델링된다. 기존의 잡음 모델 추정 체계에 사용되는 하드 분할 접근법은 추정을 위한 표본 크기를 줄임으로써 추정 정확도를 저하시키지만, 소프트 분할 방식을 통해 표본 크기를 늘리면 추정 정확도가 향상될 수 있다. 본 논문에서는 단일 이미지에서 푸아송-가우스 잡음에 대한 빠르고 실용적인 잡음 모델 추정 방법을 제시한다. 잡음이 많은 영상 히스토그램을 기반으로 미리 고정된 수의 강도 중심선을 초기화하고 가중치 합산을 통해 이미지 강도 영역의 각 강도 중심별 잡음 분산을 계산한다. 강도 중심과 유사한 강도를 가지며 낮은 경사 영역에 해당하는 화소는 높은 가중치를 가지며 그 반대의 경우는 낮게 할당된다. 합성 및 실제 영상에 대한 실험은 제안된 방법이 기존 방법보다 20배 더 빠르게 신뢰할 수 있는 잡음 추정 기능을 제공한다는 것을 보여준다.


Keywords: signal-dependent noise, poisson noise, noise estimation, noise reduction, noise model

Ⅰ. Introduction

Images are contaminated mostly during acquisition from the image sensor. The raw-data acquired by sensors undergoes many image processing stages. Estimating the characteristics of noise is vital for improving the image quality in subsequent image processing stages. Most of the denoising methods assume additive white Gaussian noise(AWGN), which can successfully model thermal and electrical noise. The variance of noise under AWGN is fixed and signal-independent. However, due to the significant size reduction of pixel units, modern high-resolution imaging sensors are more sensitive to the photon noise[1].

Therefore, images acquired with modern imaging sensors are better modeled as signal-dependent, where the variance of noise is a function of signal intensity. The objective of the signal-dependent noise estimation methods is to predict the dependency of the noise variance on signal intensity, which is formally known as the noise level function.

Generally, the existing signal-dependent noise estimation methods consist of four steps: 1) image segmentation, 2) smooth region detection, 3) noise variance estimation, and 4) fitting obtained data samples into a global parametric model. To estimate the noise variance – intensity pairs, most common practice is to cluster noisy image into several segments with nearly constant intensities[1][2].

The main assumption is that on the homogeneous regions of each segment the intensity variations are mainly caused by noise. To detect such image regions some methods utilize high pass filters[3][4], edge and texture extractors or local statistics[5][6].

In this manner, for each intensity segment a set of homogeneous patches are obtained by means of hard segmentation. To estimate unbiased noise variance, an input image is transformed into other domains, such as discrete cosine transform(DCT)[7][8], and wavelet transform[1], where noise can be distinguished from the original image information in high frequency subbands. However, for high textured images the noise level estimates of aforementioned methods are overestimated. In addition, according to the law of large number, the estimation accuracy is naturally dependent on the size of sample. Due to the hard segmentation, the number of valid candidates on some intensity segments are low. Therefore, the noise variance estimates on these intensity segments does not converge towards the expected value.

In the last step the obtained noise variance – intensity pairs are fitted into a global parametric model. Foi et al.[1] proposed to fit the estimated data into Poissonian-Gaussian noise model using iterative maximum likelihood fitting. Liu et al.[9] extended the method by using generalized signal-dependent noise model. Li et al.[7] proposed an iterative re-weighted least squares method, by taking the texture strengths of patches into account. Although these methods improve the accuracy of the noise estimation, they are computationally complex due to their iterative framework.

In this paper, we propose a fast and accurate method for image signal-dependent noise estimation. We believe, that increasing the sample size for estimation, by using soft segmentation approach instead of conventional hard segmentation, would improve the accuracy of the estimation. Based on the noisy image histogram we initialize a pre-fixed number of intensity centroids, whose occurrence is large enough[7], and directly compute the noise variance for each intensity centroid via weighted summation. Specifically, we assign the pixels, which have similar intensity with the intensity centroid and correspond to the low gradient region, with high weights and vice versa. Unlike other conventional approaches, we estimate the noise variance directly in the image intensity domain.

To fit the estimated noise variance – intensity pairs into a noise model we utilize a least mean square error approach. According to our experiments on synthetic and real noisy images, the proposed method provides fast, and reliable noise estimates.

The rest of the paper is organized as follows. In Section 2 we briefly review the image sensor noise model. The proposed method is then presented in Section 3. The accuracy and applications of the proposed method is discussed in Section 4. Finally we draw conclusions in Section 5.


Ⅱ. Noise Model

We consider generalized signal-dependent noise model[10] of the form

zx=yxγux+wx(1) 

where xX is the pixel position, z(x) is the observed noisy pixel value, y(x) is the original noise-free pixel value, γ is a parameter that controls the dependence of noise on signal, and u(x) and w(x) are two mutually independent zero-mean Gaussian variables with variances σu2 and σw2 respectively. The noise level function σ(y(x)) of generalized signal-dependent noise model can be derived from (1), and has the affine form

σyx=yx2γσu2+σw2(2) 

The Poissonian-Gaussian noise model is a special type of generalized signal-dependent noise model with γ=0.5. In this paper, we consider this noise model as it is naturally suited for the raw-data of digital imaging sensors[1]. Our objective is to estimate the noise level function parameters (σu2 and σw2) from a single raw image.


Ⅲ. Noise Model

Ideally, one can obtain true noise level function parameters if the noise variances corresponding to every pixel intensity are known. Since we have a single raw-image at our disposal, it is impossible to estimate the noise variance for every pixel intensity. In order to obtain robust estimate of the noise variance the size of the sample has to be large enough. Conventionally the image is segmented into collection of non-overlapping level sets, where each set is characterized by its mean value and allowed deviation. The noise variance on each segment is assumed constant, and is computed only on the homogeneous regions. Therefore, for highly textured images the sample size(set of homogeneous patches in a given intensity level) is small. Our approach focuses on increasing the sample size for estimation. We proceed with detailed explanation of our proposed approach.

3.1 Intensity Centroids

The objective of the first step is to initialize a pre-fixed number of intensity levels for which the noise variances are to be estimated in subsequent steps. Hereafter, we refer to these intensity levels as intensity centroids. Similar to [7], instead of using all intensity levels we choose K intensity levels whose occurrence is larger than the p-quantile ε (p = 0.5) of the noisy image histogram h

Y=yhyϵ;k=1,2,,K(3) 

where h[y] stands for the occurrence of intensity level y. In our experiments, we consider 8-bit (i.e., 256 intensity levels) grayscale images. To avoid outliers due to the clipping problem[1], we systematically discard the minimum and maximum of the dynamic range [0, 255]. From the obtained set of intensity levels, we choose a pre-fixed number K of elements with fixed step-size Δ=YK. In this manner, we form a vector of intensity centroids

Ix=YkΔ;k=1,2,,K(4) 
3.2 Weighted Noise Level Estimation

To ease the explanation of our approach we first consider estimating the noise variance of an image contaminated with AWGN with variance σw2

zx=yx+wx(5) 

Now consider the horizontal gradient Gh of the noisy image

Ghx=zx+1-zx-1(6) 

Then, on the homogeneous region the variance of the horizontal gradient can be derived as follows:

σGhx2=2σw2(7) 

In other words, the noise variance can be directly estimated by computing the variance of the gradient of homogeneous patches.

σ'w=12xXsmoGhx2-xXsmoGhx2Xsmo(8) 
σ'k= 12xXGhxWx,k2-xXGhx2Wx,kxXWx,k(9) 
y'k=xXzxWx,kxXWx,k(10) 

where Xsmo is a set of pixels that correspond to a homogeneous region. The application of equation (8) for signal-dependent noise model of the form (1) is straightforward. The image is segmented into a collection of non-overlapping level sets and the noise variance of each segment is computed using equation (8). The accuracy of the estimation, however, will depend on the accuracy of the homogeneous region detection and the sample size (|Xsmo|). To overcome this problem, we propose a weighted noise variance estimation approach, where we directly compute the noise variance – intensity pairs for each intensity centroid via weighted summation as shown in (9) and (10), where X is a set of all pixels in the image, and W(x,k) denotes the weight of a pixel xX for the estimation of the noise variance for intensity centroid index k, and σ΄(x) and y΄(x) denote the estimated noise variance – intensity pair. The setting of W(x,k) is straightforward. We assign the pixels, which have similar intensity with the intensity centroid and correspond to the low gradient region, with high weights and vice versa:

Wx,k=exp-Ik-zx2δ2exp-Gx2δ2(11) 

where I(k) is the intensity centroid, G(x) is the gradient of a pixel, and δ1 and δ2 are sensitivity parameters. The first term of (11) takes into account the intensity similarity of the pixel with the intensity centroid. In our experiments, the sensitivity parameter δ1 is set to 80. The second term takes into account the flatness of the neighborhood around the pixel. To discard the effect of the noise on weighting, in our implementation we employ smoothed horizontal, vertical and average gradients of the image as follows:

Gx=maxGh',Gv',Gavg'(12) 

The smoothed horizontal Gh' and vertical Gv' gradients are obtained by convolving the noisy image with [3×3] average operator and [3×1] gradient operators (6). The average gradient is obtained by convolving the noisy image with following operator

Gavg=1111-81111(13) 

To further improve the accuracy of our estimation we set the sensitivity parameter δ2 as the average noise variance. The average noise variance of the image is obtained by using the method presented in [11], where the variance of additive white Gaussian noise was estimated by using the difference of two Laplacians.

3.3 Noise Level Function Estimation

Let us denote the estimated noise variance – intensity pairs and noise level function parameters as vectors of the form Y=y1',y2',,yK';1,1,,1T, v=σ1',σ2',,σK'T and p=σu2,σw2, where [ㆍ]T denotes the transpose operator. Then, one can formulate the global parameter estimation problem as the least square optimization problem of the form

p'=argminpYp-v22(14) 

The problem (14) can be solved iteratively or directly by means of the closed form solution p

p'=YTY-1YTv(15) 

In our implementation, we utilized the iterative solution, as it provides more robust parameter estimation with negligible additional complexity.


Ⅳ. Experimental Results

To evaluate the accuracy of the estimation we test algorithms on the noise free image set shown in Fig. 1(a). The synthetic noisy images are generated by using three parameter settings(σu = 1.0, σw = 2.0; σu = 1.5, σw = 3.0; σu = 2.0, σw = 4.0) corresponding to low, medium and heavy contaminations, respectively. We measure the accuracy of the estimation in terms of root mean squared error(RMSE) against the theoretical curve (2).


Fig. 1. 
Image sets used in experiments

Table 1 shows the detailed comparison of estimation accuracy of algorithms. The presented RMSE results are the average of ten realizations. The average RMSE indicates that the proposed method outperforms the state-of-the-art for medium and heavy noise levels. In addition, while the estimation accuracy of all competing algorithms degrade with increasing the noise level, the proposed method is comparatively robust against noise level variations. The estimated noise level functions for ‘Traffic’ test image are depicted in Fig. 2.

Table 1. 
Comparison of estimation accuracy in terms of RMSE(×10−2) against theoretical noise level function (2) for various noise levels (Low: σu = 1.0,σw = 2.0, Medium: σu = 1.5,σw = 3.0 and Heavy: σu = 2.0,σw = 4.0)
Image Low contamination Medium contamination Heavy contamination
Pyatrykhet al.[12] Foiet al.[1] Proposed Pyatrykhet al.[12] Foiet al.[1] Proposed Pyatrykhet al.[12] Foiet al.[1] Proposed
Street1 0.13 0.31 0.16 0.23 0.31 0.28 0.29 0.34 0.33
Street2 0.08 0.30 0.30 0.23 0.40 0.33 0.26 0.44 0.36
Yard 0.23 0.39 0.07 0.29 0.38 0.11 0.27 0.48 0.03
Building1 0.09 0.29 0.06 0.21 0.39 0.04 0.57 0.53 0.01
Building2 0.11 0.29 0.16 0.24 0.35 0.11 0.45 0.54 0.31
Hallway 0.04 0.22 0.10 0.05 0.27 0.37 0.15 0.34 0.47
Plaza 0.19 0.22 0.31 0.33 0.36 0.33 0.42 0.39 0.35
Trafic 0.17 1.12 0.32 0.41 0.86 0.27 0.68 0.79 0.27
Average 0.13 0.39 0.18 0.25 0.41 0.23 0.39 0.48 0.27


Fig. 2. 
Comparison of noise level function estimation with Foi et al.[1], and Pyatykh et al.[12] against theoretical noise level function (2). Estimation is performed on ‘Traffic’ test image for (a) Heavy, and (b) Low noise levels

For high noise level, conventional approaches overestimate the noise standard deviation, while the estimated noise level function of proposed approach almost coincides with the theoretical curve. Foi et al. approach[1] fails when the noise level is low, while Pyatykh et al. approach[12] performs well(best RMSE).

In Table 2, we summarized the average RMSE results and the average runtimes of corresponding algorithms. As shown, the proposed method on average has the best accuracy and is 20 times faster than[1], and 700 times faster than[12].

Table 2. 
Comparison of RMSE and computational complexity(runtime)
Pyatykh et al. Foi et al. Proposed
RMSE(x10-2) Runtime(sec) RMSE(x10-2) Runtime(sec) RMSE(x10-2) Runtime(sec)
Low 0.13 328.2 0.39 7.51 0.18 0.42
Medium 0.25 321.8 0.41 8.84 0.23 0.44
Heavy 0.39 287.7 0.48 9.92 0.27 0.46
Average 0.26 312.6 0.43 8.75 0.23 0.44

4.2 Application in Noise Reduction

To highlight the effectiveness of the noise level function estimation, we utilize the obtained noise parameters on noise reduction. The estimated noise parameters can be applied to any AWGN noise reduction algorithm to consider signal-dependent noise.

In this paper, we consider BM3D[13]. Similar to other conventional approaches[1][14], in our implementation, we utilized variance stabilizing transformation(VST)[15]. The objective of this transformation is to transform an input signal-dependent noisy image into a domain where the noise has fixed variance(AWGN). The accuracy of VST and, consequently, the performance of the noise reduction is dependent on the accuracy of the noise parameter estimation. We evaluate the performance of the noise reduction algorithms in terms of PSNR on real noisy image set shown in Fig. 1(b).

We evaluated only fast and practical algorithms. The base for comparison is original BM3D results with default parameters. We compared the performance of our scheme with the performance of VST-BM3D based method often referred to as state-of-the-art[1].

As shown in Table 3, the incorporation of the proposed fast estimation approach has the highest PSNR among compared algorithms, and is two times faster than[1], making it suitable for practical applications.

Table 3. 
Comparison of BM3D based noise reduction performance in terms of PSNR in dB
Image Noise level BM3D[13]
Original Foi et al. Proposed
Shoes 35.11 40.96 40.83 41.09
Kitchen 32.03 36.39 39.16 39.20
Desk 26.12 33.42 33.50 33.40
Sink 33.96 41.26 42.62 43.17
Painting 33.39 36.53 37.19 37.53
Papers 34.66 36.48 37.16 37.21
Wall 27.06 33.14 34.79 35.08
Flowers 35.38 37.01 39.93 39.65
Average 32.21 36.90 38.15 38.29
Runtime (in seconds) 13.17 35.14 17.09


Ⅴ. Conclusion

We presented a fast and practical noise level function estimation method for Poissonian-Gaussian noise from a single image. The hard segmentation approach used in the conventional noise estimation schemes reduces the sample size of estimation, consequently degrading the estimation accuracy.

We showed that increasing the sample size for estimation, by using the proposed soft segmentation approach, is not only computationally efficient, but also improves the accuracy of the estimation. We initialize a pre-fixed number of intensity centroids based on the noisy image histogram, and compute the noise variances for each intensity centroid in the image intensity domain via weighted summation. The pixels, which have similar intensity with the intensity centroid and correspond to the low gradient region, are assigned with high weights and vice versa. According to our experiments on synthetic and real noisy images, our method provides fast and reliable noise estimates.


References
1. A. Foi, M. Trimeche, V. Katkovnik, and K. Egiazarian, "Practical Poissonian-Gaussian noise modeling and fitting for single-image raw-data", IEEE Transactions on Image Processing, Vol. 17, No. 10, pp. 1737-1754, Oct. 2008.
2. X. Liu, M. Tanaka, and M. Okutomi, "Signal dependent noise removal from a single image", 2014 IEEE International Conference on Image Processing(ICIP), Paris, France, pp. 2679-2683, Oct. 2014.
3. A. Amer and E. Dubois, "Fast and reliable structure-oriented video noise estimation", IEEE Transactions on Circuits and Systems for Video Technology, Vol. 15, No. 1, pp. 113-118, Jan. 2005.
4. P. Fu, Q. Sun, Z. Ji, and Q. Chen, "A new method for noise estimation in single-band remote sensing images", 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, Chongqing, China, pp. 1664-1668, May 2012.
5. X. Liu, M. Tanaka, and M. Okutomi, "Noise level estimation using weak textured patches of a single noisy image", 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA, pp. 665-668, Sep. 2012.
6. S. Pyatykh, J. Hesser, and L. Zheng, "Image noise level estimation by principal component analysis", IEEE transactions on image processing, Vol. 22, No. 2, pp. 687-699, Feb. 2013.
7. L. Dong, J. Zhou, and G. Zhai, "Efficient image sensor noise estimation via iterative re-weighted least squares", 2017 IEEE International Conference on Multimedia and Expo(ICME), Hong Kong, China, pp. 1326-1331, Jul. 2017.
8. N. N. Ponomarenko, V. V. Lukin, M. S. Zriakhov, A. Kaama, and J. Astola, "An automatic approach to lossy compression of AVIRIS images", 2007 IEEE International Geoscience and Remote Sensing Symposium, Barcelona, Spain, pp. 472-475, Jul. 2007.
9. X. Liu, M. Tanaka, and M. Okutomi, "Practical signal-dependent noise parameter estimation from a single noisy image", IEEE Transactions on Image Processing, Vol. 23, No. 10, pp. 4361-4371, Oct. 2014.
10. D. T. Kuan, A. A. Sawchuk, T. C. Strand, and P. Chavel, "Adaptive noise smoothing filter for images with signal-dependent noise", IEEE transactions on pattern analysis and machine intelligence, Vol. PAMI-7, No. 2, pp. 165-177, Mar. 1985.
11. J. Immerkaer, "Fast noise variance estimation", Computer vision and image understanding, Vol. 64, No. 2, pp. 300-302, Sep. 1996.
12. S. Pyatykh and J. Hesser, "Image sensor noise parameter estimation by variance stabilization and normality assessment", IEEE Transactions on Image Processing, Vol. 23, No. 9, pp. 3990-3998, Sep. 2014.
13. K. Dabov, A. Foi, V. Katkovnikoi, and K. Egiazarian, "Image denoising by sparse 3-D transform-domain collaborative filtering", IEEE Transactions on image processing, Vol. 16, No. 8, pp. 2080-2095, Aug. 2007.
14. M. Kakitalo and A. Foi, "Optimal inversion of the Anscombe transformation in low-count Poisson image denoising", IEEE Transactions on Image Processing, Vol. 20, No. 1, pp. 99-109, Jan. 2011.
15. J. L. Starck, F. D. Murtagh, and A. Bijaoui, "Image Processing and Data Analysis", Cambridge University Press, pp. 263-265, Oct. 2009.
16. Noise-free test images, http://mcolom.info/pages/no_noise_images/ [accessed: Oct. 23, 2018]

Authors
Hyun Sang Park

1991 : BS degree in Department of Electrical Engineering, KAIST

1993 : MS degree in Department of Electrical Engineering, KAIST

1999 : PhD degrees in Department of Electrical Engineering, KAIST

2005 ~ present : Professor in Division of Electrical, Electronic, and Control Engineering, Kongju National University

Research interests : image processing algorithms, algorithm-to-hardware mapping, system-on-a-chip architecture design