| 摘 要: 针对新闻推荐任务过度依赖用户历史行为数据可能导致的用户隐私信息泄露等问题,提出一种结合预训练模型、双向长短期记忆网络及多头注意力(RoBERTa-BiLSTM-MA)的热点新闻推荐方法。该方法利用RoBERTa和BiLSTM模型提取文本语义特征,并借助多头注意力机制捕获新闻内部的关键信息以及不同组成部分之间的关联,减少不相关信息的干扰。通过提高对新闻热度预测的准确率,达到优化推荐效果的目的。由于热点新闻推荐领域缺乏公开数据集,因此专门构建了一个中文体育新闻数据集(SPORTNEWS)。实验结果表明,在SPORTNEWS数据集上,与经典新闻推荐模型相比,RoBERTa-BiLSTM-MA在Acc、F1、NDCG@5和NDCG@10等指标上均有提升,相较于最优对比模型分别提升了1.29百分点、1.1百分点、17.14百分点和10.53百分点。 |
| 关键词: 新闻推荐;热度预测;预训练模型;多头注意力机制;深度学习 |
|
中图分类号: TP391
文献标识码: A
|
| 基金项目: 国家自然科学基金青年项目(62306056) |
|
| Hot News Recommendation Method Basedon RoBERTa-BiLSTM-MA |
|
WANG Changhao1, DU Jiaqing1, WANG Ye2,3, LIU Kai1
|
(1.School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi'an 710021, China;
2.School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
3.Key Laboratory of Computational Intelligence, Chongqing 400065, China)
wangchanghao@sust.edu.cn; Du_Jiaqing@outlook.com; wangye@cqupt.edu.cn; liukaixa@163.com
|
| Abstract: To address issues such as potential user privacy leaks caused by over-reliance on historical behavioral data in news recommendation tasks, this paper proposes a hot news recommendation method that combines a pretrained model, a Bidirectional Long Short-Term Memory Networks, and Multi-head Attention (RoBERTa-BiLSTMMA).
This method utilizes RoBERTa and BiLSTM to extract textual semantic features and employs a multi-head attention mechanism to capture key information within news articles and correlations between different components,thereby reducing interference from irrelevant information. By improving the accuracy of news popularity prediction, the method aims to optimize recommendation performance. Due to the lack of publicly available datasets in the hot news recommendation domain, a Chinese sports news dataset (SPORTNEWS) is specifically constructed. Experimental results on the SPORTNEWS dataset demonstrate that, compared to classical news recommendation models, RoBERTa-BiLSTM-MA achieves improvements in metrics such as Accuracy (Acc), F1-score (F1), NDCG@5, and NDCG@10,outperforming the best baseline model by 1.29 percentage points, 1.1 percentage points, 17.14 percentage points, and10.53 percentage points respectively. |
| Keywords: news recommendation; popularity prediction; pre-trained model; Multi-head Attention; deep learning |
