FusionGAN: A generative adversarial network for infrared and visible image fusion

Infrared and visible image fusion methods and applications: A survey

In this paper, we proposed a new end-to-end model, termed as dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Our method establishes an adversarial game between a generator and two discriminators. The generator aims to generate a real-like fused image based on a specifically designed content loss to fool the two discriminators, while the two discriminators aim to distinguish the structure differences between the fused image and two source images, respectively, in addition to the content loss. Consequently, the fused image is forced to simultaneously keep the thermal radiation in the infrared image and the texture details in the visible image. Moreover, to fuse source images of different resolutions, e.g. , a low-resolution infrared image and a high-resolution visible image, our DDcGAN constrains the downsampled fused image to have similar property with the infrared image. This can avoid causing thermal radiation information blurring or visible texture detail loss, which typically happens in traditional methods. In addition, we also apply our DDcGAN to fusing multi-modality medical images of different resolutions, e.g. , a low-resolution positron emission tomography image and a high-resolution magnetic resonance image. The qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our DDcGAN over the state-of-the-art, in terms of both visual effect and quantitative metrics. Our code is publicly available at https://github.com/jiayi-ma/DDcGAN .

DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion

https://ayhoekstra.nl/pubs/Mekonnen-Hoekstra-2014-WaterFootprintBenchmarks.pdf

Water footprint benchmarks for crop production: A first global assessment

Infrared and visible image fusion via detail preserving adversarial learning

The use of disparate data sources within a pixel level image fusion procedure has been well documented for pan-sharpening studies. The present paper explores various image fusion procedures for the fusion of multi-spectral ASTER data and a RadarSAT-1 SAR scene. The research sought to determine which fusion procedure merged the largest amount of SAR texture into the ASTER scenes, while also preserving the spectral content. An additional application based maximum likelihood classification assessment was also undertaken. Three SAR scenes were tested namely, one backscatter scene and two textural measures calculated using grey level co-occurrence matrices (GLCM). Each of these were fused to the ASTER data using the following established approaches; Brovey transformation, Intensity Hue and Saturation, Principal Component Substitution, Discrete wavelet transformation, and a modified discrete wavelet transformation using the IHS approach. Resulting data sets were assessed using qualitative and quantitative (entropy, universal image quality index, maximum likelihood classification) approaches. Results from the study indicated that while all post fusion data sets contained more information (entropy analysis), only the frequency-based fusion approaches managed to preserve the spectral quality of the original imagery. Furthermore results also indicated that the textural (mean, contrast) SAR scenes did not add any significant amount of information to the post-fusion imagery. Classification accuracy was not improved when comparing ASTER optical data and pseudo optical bands generated from the fusion analysis. Accuracies range from 68.4% for the ASTER data to well below 50% for the component substitution methods. Frequency based approaches also returned lower accuracies when compared to the unfused optical data. The present study essentially replicated (pan-sharpening) studies using the high resolution SAR scene as a pseudo panchromatic band.

Assessment of image fusion procedures using entropy, image quality, and multispectral classification

The commitment by over 100 governments covering over 90% of the world’s forests at the COP26 in Glasgow to end deforestation by 2030 requires more effective forest monitoring systems. The near real-time (NRT) change detection of forest cover loss enables forest landowners, government agencies and local communities to monitor natural and anthropogenic disturbances in a much timelier fashion than the thematic maps that are released every year. NRT deforestation alerts enable the establishment of more up-to-date forest inventories and rapid responses to unlicensed logging. The Copernicus Sentinel-2 satellites provide operational Earth observation (EO) data from multi-spectral optical/near-infrared wavelengths every five days at a global scale and at 10 m resolution. The amount of acquired data requires cloud computing or high-performance computing for ongoing monitoring systems and an automated system for processing, analyzing and delivering the information promptly. Here, we present a Sentinel-2-based NRT change detection system, assess its performance over two study sites, Manantlán in Mexico and Cartagena del Chairá in Colombia, and evaluate the forest changes that occurred in 2018. An independent validation with very high-resolution PlanetScope (~3 m) and RapidEye (~5 m) data suggests that the proposed NRT change detection system can accurately detect forest cover loss (> 87%), other vegetation loss (> 76%) and other vegetation gain (> 71%). Furthermore, the proposed NRT change detection system is designed to be attuned using in situ data. Therefore, it is scalable to larger regions, entire countries and even continents.

https://www.mdpi.com/2072-4292/14/3/707/pdf?version=1644226449

Near Real-Time Change Detection System Using Sentinel-2 and Machine Learning: A Test for Mexican and Colombian Forests

Interpreting remote sensing imagery enables numerous downstream applications ranging from land-use planning to deforestation monitoring. Robustly classifying this data is challenging due to the Earth's geographic diversity. While many distinct satellite and aerial image classification datasets exist, there is yet to be a benchmark curated that suitably covers this diversity. In this work, we introduce SATellite ImageNet (SATIN), a metadataset curated from 27 existing remotely sensed datasets, and comprehensively evaluate the zero-shot transfer classification capabilities of a broad range of vision-language (VL) models on SATIN. We find SATIN to be a challenging benchmark-the strongest method we evaluate achieves a classification accuracy of 52.0%. We provide a $\href{https://satinbenchmark.github.io}{\text{public leaderboard}}$ to guide and track the progress of VL models in this important domain.

J. Wesley Roberts

Papers

Assessment of image fusion procedures using entropy, image quality, and multispectral classification

Near Real-Time Change Detection System Using Sentinel-2 and Machine Learning: A Test for Mexican and Colombian Forests

SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models