| 摘 要: 针对生成对抗网络的图像融合任务中,因为只关注了一种图像特征,而导致信息缺失的问题。提出了一种基于类激活多尺度注意力的生成对抗网络融合算法。首先,使用类激活注意力特征机制进行特征提取;其次,在融合时使用了红外与可见特征融合和可见与红外特征融合双通道,使融合图像中红外源图像和可见光源图像的特征更加平衡。模型在TNO数据集上进行大量的对比实验,相较于同类算法,互信息提升了11.28%,标准差提升了4.18%,峰值信噪比提升了2.00%。 |
| 关键词: 注意力机制 生成对抗网络 类激活图 |
|
中图分类号: TP391
文献标识码: A
|
| 基金项目: 陕西省自然科学基础研究计划(2022JQ-175);陕西省教育厅专项科研计划(22JK0303);陕西科技大学科研启动项目(2020BJ-18) |
|
| Generative Adversarial Network for Infrared and Visible Image Fusion Based on Class Activation Multi-Scale Attention |
|
GAO Qiming, YAO Bin, WANG Meijia
|
(School of Electronic Information and Artificial Intelligence, Shaanxi University of Science & Technology, Xi’an, 710021, China)
cogrn7@163.com; yaobin@sust.edu.cn; 4672@sust.edu.cn
|
| Abstract: To address the issue of information loss in Generative Adversarial Network (GAN)-based image fusion tasks caused by focusing on only one type of image feature, this study proposes a GAN fusion algorithm incorporating
class activation mult-i scale attention. Firstly, a class activation attention feature mechanism is employed for feature extraction. Subsequently, a dua-l pathway fusion strategy-infrared-to-visible feature fusion and visible-to-infrared feature
fusion is adopted during the fusion stage. This approach ensures a more balanced representation of features from both infrared and visible source images in the fused output. Extensive comparative experiments conducted on the TNO
dataset demonstrate that, compared to state-o-f the-art methods, the proposed model achieves an 11.28% increase in Mutual Information (MI), a 4.18% improvement in Standard Deviation (SD), and a 2.00% enhancement in Peak Signa-l
to-Noise Ratio (PSNR). |
| Keywords: attention mechanism generative adversarial network (GAN) class activation map (CAM) |
