Image fusion using ICA bases

Image fusion describes the process of combining information from different sensors that capture the same scene. The objective of this task is to enhance the perception of the observed scene, which may not be achievable by a single sensor. Many approaches have been proposed with very good performance, mainly employing the Dual-Tree Wavelet Transform and various fusion rules. Here, we explore the application of ICA and Topographic ICA bases to perform image fusion using various set of rules. The bases are trained using images of similar context. To fuse the images, we use pixel-based image fusion, i.e. fusion performed in pixel-by-pixel basis. The max-abs, median and a proposed weighted combination fusion rules are compared in performance. The proposed scheme outperforms the wavelet-based methods, however, with slightly increased computational complexity.

In a multimodal fusion scenario, the images have different intensity range. Balancing or combining the intensity range of the input images to create an optimal intensity range for the fused image in terms of the Piella Index was also proposed.

This work is documented in:

" Optimal Contrast Correction for ICA-based Fusion of Multimodal Images" , IEEE Sensors Journal, Vol. 8, No. 12, pp. 2016 - 2026, Dec. 2008.
" Pixel-based and Region-based Image Fusion schemes using ICA bases ", Information Fusion 8 (2), pp. 131 - 142, April 2007.
"Region-based ICA Image Fusion using Textural Information ", 18th Int. Conf on DSP (DSP2013), July 1-3, 2013, Santorini, Greece.
"Adaptive Image Fusion using ICA bases ", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 2006, Toulouse, France.

You can find a MATLAB implementation of the first two papers here. Please cite our work, if you plan to use this code. [ code ]

Ground Truth Input 1 Input 2 Input 3
Symmlet7(max-abs) Dual-tree Wavelet (max-abs) ICA bases (max-abs) TopoICA bases (max-abs)
  TopoICA bases (mean) TopoICA bases (Weighted Combination)  

Input 1 Input 2 DT-WT maxabs ICA maxabs
TopoICA-maxabs) TopoICA-mean TopoICA-Weighted TopoICA-regional
  TopoICA-Laplacian TopoICA-Verhulstian  

Input 1 Input 2 Input 3 DT-WT maxabs
ICA maxabs TopoICA-maxabs) TopoICA-mean TopoICA-Weighted
TopoICA-Regional TopoICA-Laplacian TopoICA-Verhulstian TopoICA-Optimal