| 摘 要: 胃癌作为高发恶性肿瘤,其致死率近年来居高不下,因此精准预测胃癌患者的生存风险对于治疗至关重要。文章提出了一种基于多模态深度学习的预测模型,旨在评估胃癌患者的生存风险。该模型整合了H&E(Hematoxylin-Eosin staining)染色图像和基因表达数据,首先,采用ResNet18卷积神经网络模型提取深层H&E图像信息,将其编码为一维特征向量。其次,采用多模态紧凑型双线性池化方法,将图像特征与基因表达数据进行融合,用于预测胃癌患者的风险分数。在TCGA的胃癌样本中,该模型的一致性指数(c-index)为0.70。在测试集上进行的Kaplan-Meier分析结果显示,模型成功地区分出高风险群和低风险群。结果表明,该模型在区分胃癌患者风险层次方面表现出色,具有显著优势。 |
| 关键词: 胃癌;H&E染色图像;基因表达;深度学习;多模态 |
|
中图分类号: TP391.41
文献标识码: A
|
|
| Predicting Gastric Cancer Survival Risk with Multi-Modal Deep Learning Model Based on H&E Images and Gene Expression Dat |
|
MA Yanyu, HE Pingan
|
(School of Science, Zhejiang Sci-Tech University, Hangzhou 310018, China)
zstumayanyu@163.com; pinganhe@zstu.edu.cn
|
| Abstract: Gastric cancer, as a common malignant tumor, has a high mortality rate in recent years. Therefore, accurately predicting the survival risk of gastric cancer patients is crucial for treatment. This paper proposes a predictive model based on multi-modal deep learning to assess the survival risk of gastric cancer patients. The model integrates H&E ( Hematoxylin-Eosin staining) stained images and gene expression data. Firstly, the ResNet18 convolutional neural network model is used to extract deep H&E image information, encoding it into a one-dimensional feature vector. Secondly, a multi-modal compact bilinear pooling method is employed to merge image features with gene expression data for predicting the risk scores of gastric cancer patients. The model achieved a concordance index (c-index) of 0.70 in gastric cancer samples from TCGA. Kaplan-Meier analysis on the test set successfully differentiates between high-risk and low-risk groups. The results indicate that the model performs well in distinguishing risk levels of gastric cancer patients, demonstrating significant advantages. |
| Keywords: gastric cancer; H&E stained images; gene expression; deep learning; multi-modal |
