| 摘 要: 手语识别在帮助听障人士与健听人建立有效的沟通中发挥着重要作用。随着人工智能技术的快速发展,越来越多的深度学习方法被应用于手语识别领域。因此,提出了一种基于预训练VGG16模型和手部关键点信息的手语识别模型:Full-VGG16。该模型通过VGG16提取图像特征,并融合通过mediaPipe抽取的关键点特征来进行手语识别。实验结果表明,Full-VGG16模型在手语识别任务中表现优异,准确率达到99.52%。通过消融实验,验证了图像特征和关键点特征的有效性,显示出多模态融合方法的优势。 |
| 关键词: 手语识别 VGG16 手部关键点 MediaPipe |
|
中图分类号: TP391.41
文献标识码: A
|
|
| Research on Sign Language Recognition by Integrating VGG16 and Keypoint Information |
|
TONG Ting, WANG Shuqin, PAN Jiawen, LI Jianyu, REN Tao, HAN Jiajun
|
(School of Smart Education, Jiangsu Normal University, Xuzhou 221000, China)
3020224277@jsnu.edu.cn; 6020020070@jsnu.edu.cn; 3020224303@jsnu.edu.cn; 2823633196@qq.com; 2690060222@qq.com; 3479830281@qq.com
|
| Abstract: Sign language recognition plays a crucial role in facilitating communication between hearing-impaired individuals and hearing people. With the rapid development of artificial intelligence, an increasing number of deep learning methods are being applied to sign language recognition. This study proposes Full-VGG16, a sign language recognition model based on the pretrained VGG16 architecture fused with hand keypoint information. The model extracts image features through VGG16 and integrates keypoint features obtained via MediaPipe for sign language recognition. Experimental results demonstrate that Full-VGG16 achieves outstanding performance in sign language recognition tasks, attaining an accuracy of 99.52% . Ablation studies confirm the effectiveness of both image features and keypoint features, highlighting the advantages of multimodal fusion approaches. |
| Keywords: sign language recognition VGG16 hand keypoints MediaPipe |
