hadoop+lucene+web 综合小demo

阅读更多

很长一段时间没有关注hadoop，突然间有兴致，于是动手又研究一下

准备的基础如下:

下载hadoop-1.0.4,tomcat,lucene由于关于hadoop的基础资料也挺多的，所以这里只写一段关于hadoop+lucene+web小综合的demo设计思路， 希望对初学入门者有所启示，如下图:

 

本例子会引用最初hadoop提供的wordcount例子,具体的代码可以通过网上得到,然后运行.

workcount输入文件名称inputfile.txt 包含内容：

Hello World Bye World goole

workcount输出的文件名称part-00000 ，包含内容：

Bye 1
Hello 1
World 2
goole 1

假设:在生成part-00000 文件之后,马上调用lucene创建索引文件

创建索引文件的java代码如下：

 // @author minn

IndexWriter writer = new IndexWriter(PRE_PATH+ "index",new SimpleAnalyzer(),true);
			FileReader read = new FileReader(PRE_PATH+"part-00000");
			BufferedReader br = new BufferedReader(read);
			String row;
			Document document =null;
		    while((row = br.readLine())!=null){
				String tmp[]=row.split("\\s+");
				document=new Document();
		    	document.add(new Field("name",tmp[0],Field.Store.YES,Field.Index.TOKENIZED));
		    	document.add(new Field("count",tmp[1],Field.Store.YES,Field.Index.TOKENIZED));
				writer.addDocument(document);
			   }
			writer.optimize();
			writer.close();

 

查找java代码：

/**
 * 
 * @author minn
 *
 */
public static String searchWord(String word) throws Exception {
		Analyzer analyzer = new StandardAnalyzer();
		String result = "";
		IndexSearcher indexSearcher = null;

		indexSearcher = new IndexSearcher(PRE_PATH + "index");

		QueryParser queryParser = new QueryParser("name", analyzer);
		Query query = null;

		query = queryParser.parse(word);

		if (null != query && null != indexSearcher) {
			Hits hits = indexSearcher.search(query);

			result = hits.doc(0).get("count");

		}
		return result;
	}

 

servlet调用lucene查找：

package servlet;

import java.io.IOException;
import java.io.PrintWriter;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import lucene.SearchHadoopFile;



/**
 * Servlet implementation class HadoopServlet
 * @author minn
 */
public class HadoopServlet extends HttpServlet {
	private static final long serialVersionUID = 1L;
       

    /**
     * @see HttpServlet#HttpServlet()
     */
    public HadoopServlet() {
        super();
        // TODO Auto-generated constructor stub
    }

	/**
	 * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
	 */
	protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		// TODO Auto-generated method stub
		doPost(request,response);
	}

	/**
	 * @see HttpServlet#doPost(HttpServletRequest request, HttpServletResponse response)
	 */
	protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
		// TODO Auto-generated method stubintl
		
	 
		String result="";
		PrintWriter out=response.getWriter();
		try {
			 result= SearchHadoopFile.searchWord(request.getParameter("word"));
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		out.write(result);
		
		out.flush();
		out.close();
	}

}

 

web页面：

写道

<%@ page language="java" contentType="text/html; charset=UTF-8"
pageEncoding="UTF-8"%>
<%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %>

关键字:Hello,World bye,google

查找字段:

 

效果界面如下:

新增对这一方案的部分实现：多语言版基础管理系统展示[es6版]-简单整合spring+lucene+hadoop 

 

 

• 
• 大小: 42.6 KB

• 
• 大小: 8.4 KB

• 查看图片附件