收稿日期: 2017-10-16
修回日期: 2017-12-08
网络出版日期: 2022-05-14
版权
Design of Machine Learning Web Service Engine Based on Spark
Received date: 2017-10-16
Revised date: 2017-12-08
Online published: 2022-05-14
Copyright
夏冉 . 基于Spark的机器学习Web服务引擎设计[J]. 指挥控制与仿真, 2018 , 40(1) : 113 -117 . DOI: 10.3969/j.issn.1673-3819.2018.01.022
In addition to focusing on the learning methods and the algorithm precision in the traditional sense, machine learning needs to pay attention to the ease of use. Aiming at the ease of use, this paper proposes a machine learning web service engine that provides restful interface services externally. By encapsulating machine learning algorithms, models and optimizers, the complex parameter selection and optimization process is shielded, simplifying the use of machine learning. Finally, taking the application of spam classification and comic recommendation as an example, the training data are input structurally, and the prediction results are obtained by query to complete the mail classification and comic recommendation functions. The experimental results show that the framework can carry different machine learning modules for different applications, verify the functions of the service engine and implement the machine learning service easily.
Key words: Spark; machine learning; service engine; real-time prediction; inquiry service
[1] | Spark, Lightning-fast cluster computing.[EB/OL]. , 2011-07-01. |
[2] | J Hunt. Introduction to Akka Actors[M].Springer International Publishing, 2014: 383-398. |
[3] | 胡于响. 基于Spark的推荐系统的设计与实现[D]. 杭州:浙江大学, 2015. |
[4] | 岑凯伦, 于红岩, 杨腾霄. 大数据下基于Spark的电商实时推荐系统的设计与实现[J]. 现代计算机(专业版), 2016(24):61-69. |
[5] | 岑凯伦, 于红岩, 杨腾霄. 大数据下基于Spark的电商实时推荐系统的设计与实现[J]. 现代计算机(专业版), 2016(24):61-69. |
[6] | 刘建国, 周涛, 郭强, 等. 个性化推荐系统评价方法综述[J]. 复杂系统与复杂性科学, 2009(3):1-10. |
[7] | A Gunawardana, G Shani. A Survey of Accuracy Evaluation Metrics of Recommendation Tasks[J]. Journal of Machine Learning Research, 2009,10(10):2935-2962. |
[8] | 黄国伟, 许昱玮. 基于用户反馈的混合型垃圾邮件过滤方法[J]. 计算机应用, 2013,33(7):1861-1865. |
[9] | 马小龙. 一种改进的贝叶斯算法在垃圾邮件过滤中的研究[J]. 计算机应用研究, 2012,29(3):1091-1094 |
[10] | J Beel, S Langer, B Gipp. TF-IDuF: A Novel Term-Weighting Sheme for User Modeling based on Users’ Personal Document Collections[C]∥ Proceedings of the iConference 2017, Wuhan, China, 2017: 452-459. |
[11] | 刘嘉, 惠成峰, 都兴中, 等. 基于SaaS模式的电子商务推荐平台[J]. 计算机应用, 2012,32(9):2679-2682. |
/
〈 |
|
〉 |