Processor

When a URI is crawled, a ToeThread will execute a series of processors on it.

The processors are split into 5 distinct chains that are exectued in sequence:

Pre-fetch processing chain
Fetch processing chain
Extractor processing chain
Write/Index processing chain
Post-processing chain

你可能感兴趣的:(process)