中国科技核心期刊      中国指挥与控制学会会刊     军事装备类重点期刊
工程实践

基于Spark的机器学习Web服务引擎设计

  • 夏冉
展开
  • 江苏自动化研究所,江苏 连云港 222061
夏冉(1993-),男,湖北黄冈人,硕士研究生,研究方向为Hadoop架构、Spark架构等。

收稿日期: 2017-10-16

  修回日期: 2017-12-08

  网络出版日期: 2022-05-14

版权

指挥控制与仿真编辑部, 2018, 版权所有,未经授权,不得转载、摘编本刊文章,不得使用本刊的版式设计。

Design of Machine Learning Web Service Engine Based on Spark

  • XIA Ran
Expand
  • Jiangsu Automation Research Institute, Lianyungang 222061, China

Received date: 2017-10-16

  Revised date: 2017-12-08

  Online published: 2022-05-14

Copyright

, 2018, Copyright reserved © 2018. Office of Acta Agronomica Sinica All articles published represent the opinions of the authors, and do not reflect the official policy of the Chinese Medical Association or the Editorial Board, unless this is clearly specified.

摘要

机器学习除了要关注传统意义上的学习方法和算法精度等外,还需要关注机器学习的易用性。针对易用性,提出了一种对外提供restful接口服务的机器学习web服务引擎。通过对机器学习算法、模型和优化器等进行封装,屏蔽掉复杂的参数选取优化过程,实现简化机器学习的运用的目的。最后以垃圾邮件分类应用和漫画推荐应用为例,结构化输入训练数据,通过查询得到预测结果,完成邮件预测分类和漫画推荐功能。实验结果表明框架能搭载不同机器学习模块实现不同应用,验证了服务引擎的功能,简易的实现机器学习服务。解决了机器学习易用性的问题。

本文引用格式

夏冉 . 基于Spark的机器学习Web服务引擎设计[J]. 指挥控制与仿真, 2018 , 40(1) : 113 -117 . DOI: 10.3969/j.issn.1673-3819.2018.01.022

Abstract

In addition to focusing on the learning methods and the algorithm precision in the traditional sense, machine learning needs to pay attention to the ease of use. Aiming at the ease of use, this paper proposes a machine learning web service engine that provides restful interface services externally. By encapsulating machine learning algorithms, models and optimizers, the complex parameter selection and optimization process is shielded, simplifying the use of machine learning. Finally, taking the application of spam classification and comic recommendation as an example, the training data are input structurally, and the prediction results are obtained by query to complete the mail classification and comic recommendation functions. The experimental results show that the framework can carry different machine learning modules for different applications, verify the functions of the service engine and implement the machine learning service easily.

参考文献

[1] Spark, Lightning-fast cluster computing.[EB/OL]. , 2011-07-01.
[2] J Hunt. Introduction to Akka Actors[M].Springer International Publishing, 2014: 383-398.
[3] 胡于响. 基于Spark的推荐系统的设计与实现[D]. 杭州:浙江大学, 2015.
[4] 岑凯伦, 于红岩, 杨腾霄. 大数据下基于Spark的电商实时推荐系统的设计与实现[J]. 现代计算机(专业版), 2016(24):61-69.
[5] 岑凯伦, 于红岩, 杨腾霄. 大数据下基于Spark的电商实时推荐系统的设计与实现[J]. 现代计算机(专业版), 2016(24):61-69.
[6] 刘建国, 周涛, 郭强, 等. 个性化推荐系统评价方法综述[J]. 复杂系统与复杂性科学, 2009(3):1-10.
[7] A Gunawardana, G Shani. A Survey of Accuracy Evaluation Metrics of Recommendation Tasks[J]. Journal of Machine Learning Research, 2009,10(10):2935-2962.
[8] 黄国伟, 许昱玮. 基于用户反馈的混合型垃圾邮件过滤方法[J]. 计算机应用, 2013,33(7):1861-1865.
[9] 马小龙. 一种改进的贝叶斯算法在垃圾邮件过滤中的研究[J]. 计算机应用研究, 2012,29(3):1091-1094
[10] J Beel, S Langer, B Gipp. TF-IDuF: A Novel Term-Weighting Sheme for User Modeling based on Users’ Personal Document Collections[C]∥ Proceedings of the iConference 2017, Wuhan, China, 2017: 452-459.
[11] 刘嘉, 惠成峰, 都兴中, 等. 基于SaaS模式的电子商务推荐平台[J]. 计算机应用, 2012,32(9):2679-2682.
文章导航

/