利用PDFBOX进行PDF转成文本,可以直接生成LUCENE所需的DOCUMENT。
PDFBOX1.6中没有包含LucenePDFDocument类,需另下载pdfbox-lucene-1.6.0.jar
下载http://mvnrepository.com/artifact/org.apache.pdfbox/pdfbox-lucene/1.6.0
doc = LucenePDFDocument.getDocument(file);
单独使用
public void parsePdfToText(String file) { PDDocument doc = null; OutputStreamWriter osw = null; try { try { URL url = new URL(file); doc = PDDocument.load(url); } catch (Exception e) { try { doc = PDDocument.load(file); } catch (IOException e1) { e1.printStackTrace(); } } osw = new OutputStreamWriter( new FileOutputStream("D:\\"+System.currentTimeMillis() + ".txt"),"UTF-8"); PDFTextStripper pts = new PDFTextStripper(); pts.setSortByPosition(true); pts.setStartPage(1); pts.setEndPage(Integer.MAX_VALUE); pts.setAddMoreFormatting(true); pts.writeText(doc, osw); } catch (Exception e) { e.printStackTrace(); }finally{ if(null != osw){ try { osw.close(); } catch (IOException e) { e.printStackTrace(); } } if(null != doc){ try { doc.close(); } catch (IOException e) { e.printStackTrace(); } } } }
word,excel可用poi处理。
public static String parseWord(String path) { StringBuffer sb = new StringBuffer(); try { HWPFDocument doc = new HWPFDocument(new FileInputStream(path)); Range range = doc.getRange(); int rangeNum = range.numParagraphs(); for (int i = 0; i < rangeNum; i++) { Paragraph pg = range.getParagraph(i); sb.append(pg.text()); } } catch (Exception e) { e.printStackTrace(); } return sb.toString().trim(); }
lius是一个基于lucene的框架,其中有着许多文档转换类。
/** * * parseWordToText:word转为txt使用lius * 引入tm-extractors-0.4.jar和Lius-1.0.jar * 注意tm....jar包要放在lius前面,好像是两个包中有类的冲突问题 * (右击->BuildPath->Order and Export中进行调整),否则报java.lang.NoSuchMethodError: * org.apache.poi.poifs.filesystem.POIFSFileSystem.getRoot() * Lorg/apache/poi/poifs/filesystem/DirectoryEntry * http://stardust1900.wordpress.com/2009/12/04/poi%E7%9A%84%E4%B8%80%E4%B8%AA%E9%97%AE%E9%A2%98/ * @param @param name 设定文件 * @return void DOM对象 * @throws * @since search1.0 */ public void parseWordToText(String name){ WordIndexer wi = new WordIndexer(); File file = new File(name); try { Writer writer = new OutputStreamWriter(new FileOutputStream("D:\\a.txt")); wi.setStreamToIndex(new FileInputStream(file)); writer.write(wi.getContent()); } catch (Exception e1) { e1.printStackTrace(); } }