| 摘 要: Tensorflow Serving是Google开源的一个服务系统，针对Tensorflow Serving单体应用吞吐量低、服务调用烦琐、模型生命周期管理不完善等问题，本文设计了一种基于Tensorflow Serving的微服务软件架构方案，在部署Tensorflow Serving的Docker(开源的应用容器引擎)容器里添加本文研发的监控程序，该监控程序根据各个实例模型加载情况，将可用模型服务主动注册到微服务架构中的注册中心以实现对模型的编排管理。实验结果表明：采用本文的微服务架构方案，有效提升了Tensorflow Serving服务的吞吐量，降低了服务响应时间，简化了模型调用流程，从而满足Tensorflow Serving在生产环境中部署和运维的实际需求。 |
| 关键词: Tensorflow Serving；微服务；第三方注册 |
|
中图分类号: TP311
文献标识码: A
|
| 基金项目: 浙江省基础公益研究计划项目(LGG20F020016)；浙江省重点研发计划项目(2020C03104). |
|
| Deep Learning Model Service System based on Microservice |
|
JIANG Ningyuan, ZHANG Huaxiong
|
(School of Information, Zhejiang Sci -Tech University, Hangzhou 310018, China)
240662137@qq.com; zhxhz@zstu.edu.cn
|
| Abstract: Tensorflow Serving, an open source service system of Google, has problems of low throughput of Tensorflow Serving single application, cumbersome service invocation, and imperfect model lifecycle management. Aiming at these problems, this paper proposes to design a microservice software architecture solution based on Tensorflow Serving. Monitoring program developed in this research is added to Docker (open source application container engine) container where Tensorflow Serving is deployed. The monitoring program actively registers available model services to the registry of the microservice architecture according to loading status of each instance model, so to realize orchestration management of the model. Experimental results show that the proposed microservice architecture solution effectively improves throughput of Tensorflow Serving services, reduces service response time, and simplifies model invocation process. It can meet actual needs of Tensorflow Serving deployment, operation, and maintenance in production environment. |
| Keywords: Tensorflow Serving; microservice; third-party registration |
