Dissecting The Nutch Crawler -Factory classes: Overview

 

 

     英文原文出处：DissectingTheNutchCrawler

  转载本文请注明出处：http://blog.csdn.net/pwlazy

Factory classes: Overview

> Class net.nutch.parser.ParserFactory
> used by:
> - net.nutch.db.WebDBInjector
> - net.nutch.fetcher.Fetcher
> - net.nutch.parser.ParserChecker
>
> Class net.nutch.protocol.ProtocolFactory
> used by:
> - net.nutch.fetcher.Fetcher
> - net.nutch.parser.ParserChecker
>
> Class net.nutch.net.URLFilterFactory
> used by:
> - net.nutch.db.WebDBInjector
> - net.nutch.tools.UpdateDatabaseTool
>
> Class net.nutch.plugin.PluginRepository: used by (Parser/Protocol)Factory

Nutch's ParserFactory and ProtocolFactory classes are the key extension points for the crawler. URLFilterFactory additionally provides an extension point for other components, including WebDBInjector and UpdateDatabaseTool. These "Factory" classes can all be reconfigured by editingXML config files. So before we describe the mechanics of any of the Factory classes, we need take a quick look at Nutch's configuration system.

 

工厂类概览

net.nutch.parser.ParserFactory  被以下几个类使用

• net.nutch.db.WebDBInjector
•   net.nutch.fetcher.Fetcher
•   net.nutch.parser.ParserChecker

net.nutch.protocol.ProtocolFactory 被以下几个类使用

•  net.nutch.fetcher.Fetcher
• net.nutch.parser.ParserChecker

 net.nutch.net.URLFilterFactory 被以下几个类使用

•  net.nutch.db.WebDBInjector
•   net.nutch.tools.UpdateDatabaseTool

net.nutch.plugin.PluginRepository: 被 (Parser/Protocol)Factory 使用

对于crawler来说 ParserFactory 和 ProtocolFactory 是关键的扩展点

URLFilterFactory 又另外为其他组件(比如WebDBInjector 和 UpdateDatabaseTool)提供了一个扩展点.这些工厂类可以通过编辑xml配置文件重新配置。所以在我们阐述任何一个工厂类的机制之前，我们需要迅速浏览一下nutch的配置系统