Underwater images suffer from the problem of poor color contrast and poor visibility. These problems occurred due to the scattering of light and refraction of light while entering from rarer to denser medium. The problems result in limited usage of the images. In terms of detection, the objects in the image hardly seen and not detectable while having low possibility of tracing. A Field Programmable Gate Array (FPGA) is one such reconfigurable hardware, offering superior features than DSP & other hardware device due to their product reliability & maintainability advantages in digital image processing. FPGA has a large impact in image and video processing; this is due to the potential of the FPGA to have parallel and high computational density as compared to a general purpose microprocessor. This paper proposes underwater images enhancement by wavelet decomposition based image fusion implementation on FPGA. The color corrected and contrast-enhanced images are fused which are withdrawn from an original underwater image.
Underwater image; Color correction; Contrast enhancement; Wavelet decomposition; Image fusion;
Nowadays, the area of research has increased its interest in the marine class. But to work on the aquatic objects, it is imperative to get the clear images of the underwater object. As the air interface deals with the environmental and camera problems like dust particles, natural light, reflection, focus and distance, underwater images are also facing the same challenges. The quality of the underwater image depends on the density of water, depth of water, distance between camera and object, artificial light, water particles, etc. the depth of water increases, the water becomes denser due to sand, planktons and minerals. Due to increased density, the camera light gets reflected and deflected by particles for sometimes before reaching towards the camera and some part of camera light get absorbed by the particles. This scattering effect results in reduced visibility of image with low contrast. Secondly, the color change effect depends on the wavelength of light travel in the water.
Fusion is an important technique within many disparate fields such as remote sensing, robotics, and medical applications. Image fusion algorithm based wavelet transform is that, the two images to be processed are sampled to the one with the same size and they are respectively decomposed into the sub images using forward wavelet transform, which have the same resolution at the same levels and different resolution among different levels; and information fusion is performed based on the high-frequency sub images of decomposed images; and finally the result image is obtained using inverse wavelet transform. The result of image fusion is a single image which is more suitable for human and machine perception or further image-processing tasks.
Most of the image enhancement implementations found in literature are based on MATLAB and C/C++. MATLAB is a high performance language for technical computing and excellent tool for algorithm development and data analysis. Reconfigurable hardware in the form of FPGA is considered as a practical way of obtaining high performance for computationally intensive image processing algorithms. FPGA’s have been traditionally configured using Hardware Description Languages (HDL) Verilog and Very High Speed Integrated Circuits (VHSIC) HDL (VHDL). C-based HDL shave also been used . Another area where research is ongoing is to develop and employ high level design tools which will bring down the development time required for deploying signal processing solutions using FPGA. Xilinx System Generator (XSG) is one such tool that enables the use of Math works model based design environment Simulink for FPGA design.
We have done literature survey on underwater image and conclude that the hybridization of algorithms is done for better visualization like wavelet fusion and contrast enhancement, improving contrast and color correction etc.
J. Wang, et al. 1, proposed The image fusion method is mainly divided as three ways: the first is a direct fusion method, which is used to fuse two source images of spatial registration into an image using some simple processions such as direct selecting or weighted average. The second algorithm is based on pyramid decomposition and reconstruction, which is eventually formed through reconstruction. The third method is the fusion algorithm based on the wavelet transform, which fuses images pertinently in the feature fields of each layer using multi-resolution analysis and Mallat fast algorithm. Due to the virtue of its multi-resolution, directivity and non-redundancy, wavelet transform has been applied in image processing field successfully.
Alex, et al. 2, proposed on adaptive histogram equalization technique to improve the enhancement of images. In the adaptive histogram equalization technique, the pixels are mapped based on it local gray scale distribution. In this method, the enhancement mapping applied to a particular pixel is a function of the intensity values of pixels immediately surrounding the pixel. Hence the number of times that this calculation should be repeated is the same as the number of pixels in the image. They have implemented their algorithm on FPGA for hardware implementation. They are improving the performance by doing the parallel processing. The algorithm is implemented in Xilinx Spartan 3AM on Altium Nanoboard NB3000 board using Altium Designer.
Xiu Li, et al. 3, proposed two parameters due to underwater images quality degraded. These are light scattering and color distortion. Also, they defined that the light scattering occurs due to light be reflected and deflected number of times by the suspended particles in the water and color distortion due to absorption degrees and its vary according to the wavelength. They proposed a novel technique based on dark channel prior and luminance adjustment. Their technique resolves these
C. Ancuti, et al. 4, proposed Classical image enhancement techniques have been modified to adapt to the underwater imaging. These methods do not depend on physical modeling of underwater scenario. The most popular method is underwater image and video enhancement using fusion to combine different weighted images using saliency, luminance, and chrominance via filtering. This was the first recorded work for the enhancement of underwater images using fusion approach based on Laplacian pyramid. The authors also validated the selection of white balancing algorithm for underwater images. Although the contrast of the output images appears increased, the problem associated with it is, as reflected in the results section, the processed images are not uniformly enhanced and does not appear natural.
M. S. Hitam, et al. 5, There are many image-base methods in underwater Image enhancement. Global and local image contrast Enhancement is widely used to improve the appearance of Underwater images. Hitam et al., proposed a method called Mixture contrast limited adaptive histogram equalization (clahe). Clahe is operated on rgb and hsv color models, And the two results are combined together with Euclidean Norm. Ahmad et al. A. S. A. Ghani and n. A. M. Isa 6, proposed a new method called dual image Rayleigh-stretched contrast-limited adaptive histogram Specification, which integrated global and local contrast correction.
Yafei Wang, et al. 7, proposed fusion process involves two inputs which are represented as color corrected and contrast enhanced images extracted from original underwater image. Both the color corrected and contrast enhanced images are decomposed into low frequency and high frequency components by three-scale wavelet operator. The low frequency and high frequency components are fused via a multiscale fusion process. The low frequency components are fused by weighted average, and the high frequency components are fused by local variance. These fused low frequency and high frequency components can be reconstructed as final enhanced image. In this paper, an efficient fusion-based underwater image enhancement approach using wavelet decomposition is presented. The experimental results demonstrate that the proposed approach effectively improves the visibility of underwater images and can be utilized in image matching application for underwater environments.
The fusion strategies used in spatial domain from the previous different methods, a fusion-based underwater image enhancement using FPGA is proposed in the frequency domain. Here low frequency and high-frequency components are decomposed from the color corrected image and contrast-enhanced image by wavelet to employ the fusion process.
Fig.1 proposed block diagram of underwater image enhancement by wavelet decomposition
The proposed enhancing strategy consists of three main steps: color correction (first input of fusion process) in Section II-A, contrast enhancement (second input of fusion process) in Section II-B and multi-scale fusion process for the two inputs in Section II-C.
In preprocessing, the image will be resized to the fixed dimension. after resize of the image in this we will check the color space of input image whether it is grey or color and according to features extraction of preprocessing image will be done.
For color correction, the image is converted from RGB (Red-Green-Blue) to HSV (Hue-Saturation-Value) color space.
In HSV color space the histogram of the Value component is stretched over the whole range. This operation improves the brightness of the available colors in the image. Then the Hue and Saturation are concatenated with the corrected value component and hence the image is converted back to the RGB color space. In RGB color space, once again the histogram is stretched over the whole range (0 to 255) to achieve the color correction in all three channels. The histogram stretching is based on the mathematical expression given in the Eq. (1).
where and are the pixels of output and input images respectively. , , and are minimum and maximum values of intensities for input and output images respectively. It overcomes the limitations of underwater environments, removes the color casts and produces a natural appearance of the underwater image. It overcomes the limitations of underwater environments, removes the color casts and produces a natural appearance of the underwater image.
To enhance the contrast of the underwater images, we adopted the contrast limited adaptive histogram equalization (CLAHE). CLAHE differs from ordinary AHE in its contrast limiting. The CLAHE introduced clipping limit to overcome the noise amplification problem. The CLAHE limits the amplification by clipping the histogram at a predefined value before computing the Cumulative Distribution Function (CDF). In CLAHE technique, an input original image is divided into non-overlapping contextual regions called sub-images, tiles or blocks. The CLAHE has two key parameters: Block Size (BS) and Clip Limit (CL). These two parameters mainly control enhanced image quality. The image is getting bright when CL is increased because input image has very low intensity and larger CL makes its histogram flatter. As the BS is bigger, the dynamic range becomes larger and the contrast of image is also increasing. The two parameters determined at the point with maximum entropy curvature produce subjectively good quality of image with using the entropy of image.
The CLAHE method applies histogram equalization to each contextual region. The original histogram is clipped and the clipped pixels are redistributed to each gray level. The redistributed histogram is different with ordinary histogram, because each pixel intensity is limited to a selected maximum. But the enhanced image and the original image have the same minimum and maximum gray values.
Multi-scale Fusion Process
In the proposed fusion strategy (shown as Fig.1), color corrected image and contrast enhanced image are the first and second inputs respectively. By applying the wavelet operator each input is decomposed into low frequency and high-frequency components. Then, different fusion principles are utilized to fuse the low frequency and high frequency components. Weighted average is enforced to fuse low frequency components, while local variance is employed in the fusion of high frequency components. The new low frequency and high frequency components are generated, After the fusion process. Finally, the enhanced image is obtained by reconstructing the new low frequency and high frequency components.
Decomposition, Fusion and Inverse Composition
The wavelet based fusion algorithm consists of a sequence of low pass and high pass filter banks that are used to eliminate unwanted low and high frequencies present in the image and to acquire the detail and approximation coefficients separately for making the fusing process convenient. Fig. 2, one level, 2- dimensional decomposition of the input image into its detail and approximation coefficients is described. Each input image is filtered and down-sampled. The factor of 2 in the algorithm is used to divide the information contained in the input signal into two equal parts at each step of filtering so that the information can be analyzed deeply. In Fig. 2, only one level decomposition is shown. There are two steps in level one; the first step is achieved by applying the low pass and high pass filters with down-sampling on the rows of the input image x(r, c). This generates horizontal approximations and horizontal details respectively. In the next step, the columns in the horizontal coefficients are filtered and down-sampled into four sub images Approximate (LL), Vertical detail (LH), Horizontal detail (HL), and the Diagonal detail (HH) as shown in Fig. 2.
Figure 2: One-level-2D wavelet-based image decomposition
At the second level of decomposition, the decomposed approximate image (LL) of the first level becomes the input image and the process is repeated to scale down coefficients as shown in Fig. 2. Each input image is decomposed into its wavelet coefficients by using the procedure as described above. In our case both enhanced images: the color corrected and the contrast enhanced versions of the input image are decomposed into their wavelet coefficients then both decompositions are fused by using coefficients of maximum values as shown in Fig. 3.
Figure 3: – Fusion of one level decomposed wavelet coefficients
After combining coefficients of both enhanced images into fused coefficients, the inverse composition is applied to get the synthesized image. For the inverse composition, the reverse process is carried out with the help of up-sampling and filtering steps using filter banks to get a synthesized or enhanced image y(r,c), see Fig. 4. Since we are dealing with discrete data sets
Figure 4:- One-level-2D wavelet-based inverse composition
Figure 5: Two-level-2D decomposition, fusion of coefficients and image
so in digital image processing, each input image is decomposed into its coefficients and inversely composed into a synthesized image by using discrete wavelet transform (DWT) and Inverse discrete wavelet transform (IDWT) respectively. In Fig. 5, a complete picture of two level discrete wavelet-based decompositions, fusion and inverse composition of enhanced image is shown.
The underwater images quality degraded due to scattering of light, refraction and absorption parameters. To resolve these issues and to improve the quality of an underwater image, a number of techniques are proposed in recent years. We have done literature survey on the underwater image and conclude that the hybridization of algorithms is done for better visualization like wavelet fusion and contrast enhancement, improving contrast and color correction, etc. A review of underwater image enhancement is presented covering basic enhancement technique, issues and challenges and existing techniques for underwater image enhancement. This paper presents the implementation of underwater image enhancement on FPGA based on fusion by wavelet decomposition.
1 J. Wang, D. Zion, C. Armenakis, et al: A comparative analysis of image fusion methods, IEEE Transaction on Geosciences and Remote Sensing. Vol. 43(6) (2005), pp. 1391-1402
2Alex Raj S., Deepa, and Supriya M.H., “Underwater image enhancement using CLAHE in a reconfigurable platform,” MTS/IEEE Monterey Oceans, December 2016.
3Xiu Li, Zhixiong Yang, and Min Shang, “Underwater image enhancement via dark channel prior and luminance adjustment,” Shanghai Ocean, April 2016.
4 C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert, “Enhancing Underwater Images and Videos by Fusion”, In: Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Providence, RI, pp.81-88, 2010.
5 m. S. Hitam, e. A. Awalludin, w. N. J. H. W. Yussof and z. Bachok, Mixture contrast limited adaptive histogram equalization for underwater Image enhancement. In proc. International conference on computer Applications technology (iccat), pages 1-5, 2013.
6 a. S. A. Ghani and n. A. M. Isa, enhancement of low quality underwater Image through integrated global and local contrast correction. Applied Soft computing, 37: pages 332-344, 2015.
7Yafei Wang, Xueyan Ding, Ruoqian Wang, Jun Zhang, Xianping Fu “Fusion-based underwater image enhancement by wavelet decomposition” IEEE International Conference on Industrial Technology (ICIT) ,1013 – 1018,2017.
8 H. Wang, J. Peng, W. Wu: Fusion algorithm for multi-sensor images based on discrete Biorthogonal wavelets transforms, IEEE Proceedings on Vision. Image and Signal Processing. Vol. 149(5) (2002), pp. 283-289
9 M. X. Li, H. P. Mao, Y. C. Zhang, et al: Fusion algotithm for muti-sensor images based on PCA and lifting wavelet transformation, New Zealand Journal of Agricultural Research. Vol. (50) (2007), pp. 667-671
10 W. Z. Shi, C. Q. Zhu, Y. Tian, et al: Wavelet-based image fusion and quality assessment, Internation Journal of Applied Earth Observation and Geoinformation.Vol. 6(3-4) (2005), pp. 241-251
11 Jintasuttisak, T.; Intajag, S. Color Retinex Image Enhancement by Rayleigh Contrast Limited Histogram Equalization. International Conference on Control, Automation and Systems.2014,10,692-697.
12 Min, B.S.; Lim, D.K.; Kim, S.J.;Lee, J.H. A Novel Method of Determining Parameters of CLAHE Based on Image Entropy. International Journal of Soft Engineering and its Applications. 2013, 7, 113-120.
13 Xu, Z.; Liu, X.; Ji, N. Fog Removal from Color Images using Contrast Limited Adaptive Histogram Equalization. International Congress on Image and Signal Processing. 2009,10,1-5.
14 Smt G. Mamatha, Dr V. Sumalatha, Dr M.V. Lakshmaiah FPGA Implementation of Satellite Image Fusion Using Wavelet Substitution Method. 2015 , IEEE Science and Information Conference (SAI), 1155 – 1159