| 摘 要: 大数据时代下，人们热衷于在社交媒体上以图文等结合的方式发布状态数据。为了更好地整合海量社交媒体数据，首先，介绍了单模态、多模态、跨模态数据融合的最新研究进展，并在此基础上比较其优势与存在的问题；其次，在跨模态应用方面，对跨模态图文检索、跨模态推荐系统、跨模态情感分析、跨模态人机对话系统及其目前存在的问题进行了分析与总结；最后，得出跨模态数据融合能够实现数据合理有效利用的结论，并提出对跨模态数据融合未来发展的展望。 |
| 关键词: 单模态融合；多模态融合；跨模态融合；深度学习 |
|
中图分类号: TP301
文献标识码: A
|
| 基金项目: 河北省教育厅科学研究计划项目(重点项目，ZD2019017). |
|
| Overview of Cross-Modal Data Fusion |
|
QI Huaying, HE Ping
|
(School of Information Technology, Hebei University of Economics and Business, Shijiazhuang 050061, China )
2227568724@qq.com; pinghe@heuet.edu.cn
|
| Abstract: In the big data era, people are keen to release status data by means of pictures and texts on social media. In order to better integrate massive social media data, this paper first introduces the latest research progress of single-mode, multi-mode and cross-mode data fusion, and compares their advantages and existing problems on this basis. Secondly, in terms of cross-modal application, the cross-modal image and text retrieval, recommendation system, sentiment analysis, human-machine dialogue system, and the existing problems are analyzed and summarized. Finally, it is concluded that crossmodal data fusion can realize reasonable and effective data utilization, and the future development prospect of cross-modal data fusion is proposed. |
| Keywords: single-mode fusion; multi-modal fusion; cross-modal fusion; deep learning |
