用意: 网络上有很多关于使用mahout搭建推荐系统的文章,但是还没有一个从建立推荐系统原型至部署到简单服务器的完整教程. 虽然部分朋友对推荐系统很感兴趣, 但是因hadoop的复杂而却步. 同时对于那些没有任何Web开发经验的朋友来说, 一个完整的小型推荐系统可以很大的激发学习的兴趣和动手的冲动. 我觉得动手的冲动比看书的冲动要重要的多.
原型分为两个系列 : JAVA原型和Python原型.
这篇博客主要是介绍JAVA推荐系统原型: 主要参考[1]
使用MyEclipse和Mahout开发一个REST风格[3]的简单推荐系统
Library见下图 core server client json.
<servlet-mapping>
<servlet-name>JAX-RS REST Servlet</servlet-name>
<url-pattern>/test/*</url-pattern>
</servlet-mapping>
其中的url-pattern将是WEB路径的一部分,具体见下文.
2. 在/src目录下加入HelloRS文件. 内容如下
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
|
import
javax.ws.rs.GET;
import
javax.ws.rs.Path;
import
javax.ws.rs.Produces;
import
javax.ws.rs.core.MediaType;
//设置路径为http://域名:端口/ConTextRootURL/url-pattern + /hello
//以我的机子为例:http://localhost:8080/rs/rest/hello
@Path
(
"/recommend"
)
public
class
HelloRS {
// 这个方法将返回普通文本
@GET
@Produces
(MediaType.TEXT_PLAIN)
public
String sayPlainTextHello() {
return
"Hello REST"
;
}
// 这个方法将返回XML文件
@GET
@Produces
(MediaType.TEXT_XML)
public
String sayXMLHello() {
return
"<?xml version=\"1.0\"?>"
+
"<hello> Hello REST"
+
"</hello>"
;
}
// 这个方法将返回HTML文件
@GET
@Produces
(MediaType.TEXT_HTML)
public
String sayHtmlHello() {
return
"<html> "
+
"<title>"
+
"Hello REST"
+
"</title>"
+
"<body><h1>"
+
"Hello REST"
+
"</body></h1>"
+
"</html> "
;
}
}
|
2. 在src下新建一个RecommenderIntro.java文件,代码如下:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
|
<strong>
import
org.apache.mahout.cf.taste.impl.model.file.*;
import
org.apache.mahout.cf.taste.impl.neighborhood.*;
import
org.apache.mahout.cf.taste.impl.recommender.*;
import
org.apache.mahout.cf.taste.impl.similarity.*;
import
org.apache.mahout.cf.taste.model.*;
import
org.apache.mahout.cf.taste.neighborhood.*;
import
org.apache.mahout.cf.taste.recommender.*;
import
org.apache.mahout.cf.taste.similarity.*;
import
java.io.*;
import
java.util.*;
class
RecommenderIntro {
private
FileDataModel model;
private
PearsonCorrelationSimilarity similarity;
private
NearestNUserNeighborhood neighborhood;
private
GenericUserBasedRecommender recommender;
// 从filename中读取数据(用户id, 物品id, 评分rate), 生成数据类model, 相似类similarity以及最相近的邻居类(2个)
public
RecommenderIntro(String filename)
throws
Exception {
model =
new
FileDataModel(
new
File(filename));
similarity =
new
PearsonCorrelationSimilarity(model);
neighborhood =
new
NearestNUserNeighborhood(
2
, similarity, model);
recommender =
new
GenericUserBasedRecommender(
model, neighborhood, similarity);
}
// 对用户userid推荐前num个物品.
public
List<RecommendedItem> SimpleRecommend(
int
userid,
int
num)
throws
Exception {
List<RecommendedItem> recommendations =
recommender.recommend(
1
,
1
);
return
recommendations;
}
}</strong>
|
代码介绍:
本算法是最简单的基于用户的协同过滤. 现实解释:你想别人给你推荐一个电影,你会从一堆人中找到与你最熟悉的几个人推荐电影给你,然后找到被推荐次数最多的电影. Model类用来存储数据, mahout为了节约内存, 数据结构设计的很好,下次找个机会聊聊. Similarity计算两个人之间的相似性, 而Neighborhood则是为每个人保存最相似的2个人.最后recommener结合model\neighborhood和similarity 来为某个user推荐N个好友.
3. 修改helloRS文件修改如下所示:
将intro.csv文件放到src/目录下
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
|
<strong>
import
java.util.List;
import
javax.ws.rs.DefaultValue;
import
javax.ws.rs.GET;
import
javax.ws.rs.Path;
import
javax.ws.rs.Produces;
import
javax.ws.rs.QueryParam;
import
javax.ws.rs.core.MediaType;
import
org.apache.mahout.cf.taste.recommender.RecommendedItem;
//Sets the path to base URL + /hello
@Path
(
"/recommend"
)
public
class
HelloRS {
private
RecommenderIntro recommender =
null
;
private
String filename =
null
;
// This method is called if TEXT_PLAIN is request
@GET
@Produces
(MediaType.TEXT_PLAIN)
public
String sayPlainTextHello(
@DefaultValue
(
"1"
)
@QueryParam
(
"id"
) String id,
@DefaultValue
(
"1"
)
@QueryParam
(
"num"
) String num)
throws
Exception {
int
userId = Integer.valueOf(id);
int
rankNum = Integer.valueOf(num);
String resultStr = getRecommender(userId, rankNum);
return
resultStr;
}
// This method is called if XML is request
@GET
@Produces
(MediaType.TEXT_XML)
public
String sayXMLHello(
@DefaultValue
(
"1"
)
@QueryParam
(
"id"
) String id,
@DefaultValue
(
"1"
)
@QueryParam
(
"num"
) String num)
throws
Exception {
int
userId = Integer.valueOf(id);
int
rankNum = Integer.valueOf(num);
String resultStr = getRecommender(userId, rankNum);
return
"<?xml version=\"1.0\"?>"
+
"<hello> "
+ resultStr +
"</hello>"
;
}
// This method is called if HTML is request
@GET
@Produces
(MediaType.TEXT_HTML)
public
String sayHtmlHello(
@DefaultValue
(
"1"
)
@QueryParam
(
"id"
) String id,
@DefaultValue
(
"1"
)
@QueryParam
(
"num"
) String num)
throws
Exception {
System.out.println(id +
" "
+ num);
int
userId = Integer.valueOf(id);
int
rankNum = Integer.valueOf(num);
String resultStr = getRecommender(userId, rankNum);
return
"<html> "
+
"<title>"
+
"Hello REST"
+
"</title>"
+
"<body><h1>"
+ resultStr +
"</body></h1>"
+
"</html> "
;
}
private
String getRecommender(
int
userId,
int
num)
throws
Exception {
if
(filename ==
null
) {
String classPath =
this
.getClass().getClassLoader()
.getResource(
"/"
).getPath();
classPath = classPath.replace(
"%20"
,
"\\ "
);
filename = classPath +
"intro.csv"
;
System.out.println(filename);
}
if
(recommender ==
null
)
recommender =
new
RecommenderIntro(filename);
List<RecommendedItem> recommendedList = recommender.SimpleRecommend(
userId, num);
String resultStr =
"Result="
+ recommendedList.get(
0
).getItemID() +
" "
+ recommendedList.get(
0
).getValue();
return
resultStr;
}
}</strong>
|
代码介绍: 代码提供了XML\HTML和普通文本三个格式, 以浏览器默认的HTML格式为例.
如果浏览器 输入 http://localhost:8080/rs/rest/recommend?id=1&num=1
参数表@DefaultValue("1") @QueryParam("id") String id, @DefaultValue("1") @QueryParam("num") String num表示获得参数
id = "1", num = "1". 之后通过getRecommender来初始化Recommender并获得数据. QueryParam表示GET方法的数据.
注: 由于intro.csv数据集比较少,所有部分id和num值无法返回合适的结果.
注: 由与intro.csv最终会部署在tomcat上,所以需要获得tomcat中class的路径.
注: recommender作为成员函数,保证每一个函数都引用同一份数据,保证一致性.
获取路径的方法如下:
|
1
2
|
<strong>String classPath =
this
.getClass().getClassLoader()
.getResource(
"/"
).getPath();<span style=
"line-height:1.428571em;font-family:'sans serif', tahoma, verdana, helvetica;font-size:10pt;font-weight:normal;"
></span></strong>
|
4. 运行代码,即可使用http://localhost:8080/rs/rest/recommend?id=1&num=1 即可在浏览器中访问.
返回:
Result=104 4.257081
[1] Sean Owen "Mahout in Action" http://book.douban.com/subject/4893547/
[2] Lars Vogel REST with Java (JAX-RS) using Jersey - Tutorial http://www.vogella.com/articles/REST/article.html[3] REST 参考豆瓣API http://developers.douban.com/wiki/?title=movie_v2#reviews