很长一段时间没有关注hadoop，突然间有兴致，于是动手又研究一下
准备的基础如下:
下载hadoop-1.0.4,tomcat,lucene由于关于hadoop的基础资料也挺多的，所以这里只写一段关于hadoop+lucene+web小综合的demo设计思路， 希望对初学入门者有所启示，如下图:
本例子会引用最初hadoop提供的wordcount例子,具体的代码可以通过网上得到,然后运行.
workcount输入文件名称inputfile.txt 包含内容：
Hello World Bye World goole
workcount输出的文件名称part-00000 ，包含内容：
Bye 1
Hello 1
World 2
goole 1
假设:在生成part-00000 文件之后,马上调用lucene创建索引文件
创建索引文件的java代码如下：
// @author minn
IndexWriter writer = new IndexWriter(PRE_PATH+ "index",new SimpleAnalyzer(),true);
FileReader read = new FileReader(PRE_PATH+"part-00000");
BufferedReader br = new BufferedReader(read);
String row;
Document document =null;
while((row = br.readLine())!=null){
String tmp[]=row.split("\\s+");
document=new Document();
document.add(new Field("name",tmp[0],Field.Store.YES,Field.Index.TOKENIZED));
document.add(new Field("count",tmp[1],Field.Store.YES,Field.Index.TOKENIZED));
writer.addDocument(document);
}
writer.optimize();
writer.close();
查找java代码：
/**
*
* @author minn
*
*/
public static String searchWord(String word) throws Exception {
Analyzer analyzer = new StandardAnalyzer();
String result = "";
IndexSearcher indexSearcher = null;
indexSearcher = new IndexSearcher(PRE_PATH + "index");
QueryParser queryParser = new QueryParser("name", analyzer);
Query query = null;
query = queryParser.parse(word);
if (null != query && null != indexSearcher) {
Hits hits = indexSearcher.search(query);
result = hits.doc(0).get("count");
}
return result;
}
servlet调用lucene查找：
package servlet;
import java.io.IOException;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import lucene.SearchHadoopFile;
/**
* Servlet implementation class HadoopServlet
* @author minn
*/
public class HadoopServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
/**
* @see HttpServlet#HttpServlet()
*/
public HadoopServlet() {
super();
// TODO Auto-generated constructor stub
}
/**
* @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
*/
protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stub
doPost(request,response);
}
/**
* @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
*/
protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// TODO Auto-generated method stubintl
String result="";
PrintWriter out=response.getWriter();
try {
result= SearchHadoopFile.searchWord(request.getParameter("word"));
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
out.write(result);
out.flush();
out.close();
}
}
web页面：
写道
<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>hadoop lucene test</title>
<script type="text/javascript" src="js/jquery-1.7.1.min.js"></script>
<script type="text/javascript">
function queryWord(){
var word=$('#searchWord_id').val();
$.ajax({
type: "POST",
url: "hServlet",
data: { word: word }
}).done(function( msg ) {
$('#count_id').text('结果:'+msg);
});
}
</script>
</head>
<body>
<div>关键字:Hello,World bye,google</div>
查找字段:<input id="searchWord_id" type="text"><input type="button" onclick="queryWord()" value="查找">
<div id="count_id"></div>
</body>
</html>
pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>hadoop lucene test</title>
<script type="text/javascript" src="js/jquery-1.7.1.min.js"></script>
<script type="text/javascript">
function queryWord(){
var word=$('#searchWord_id').val();
$.ajax({
type: "POST",
url: "hServlet",
data: { word: word }
}).done(function( msg ) {
$('#count_id').text('结果:'+msg);
});
}
</script>
</head>
<body>
<div>关键字:Hello,World bye,google</div>
查找字段:<input id="searchWord_id" type="text"><input type="button" onclick="queryWord()" value="查找">
<div id="count_id"></div>
</body>
</html>
效果界面如下:
