| Package | Description |
|---|---|
| edu.uci.ics.crawler4j.crawler | |
| edu.uci.ics.crawler4j.robotstxt |
| Modifier and Type | Field and Description |
|---|---|
protected PageFetcher |
CrawlController.pageFetcher |
| Modifier and Type | Method and Description |
|---|---|
PageFetcher |
CrawlController.getPageFetcher() |
| Modifier and Type | Method and Description |
|---|---|
void |
CrawlController.setPageFetcher(PageFetcher pageFetcher) |
| Constructor and Description |
|---|
CrawlController(CrawlConfig config,
PageFetcher pageFetcher,
RobotstxtServer robotstxtServer) |
| Modifier and Type | Field and Description |
|---|---|
protected PageFetcher |
RobotstxtServer.pageFetcher |
| Constructor and Description |
|---|
RobotstxtServer(RobotstxtConfig config,
PageFetcher pageFetcher) |
RobotstxtServer(RobotstxtConfig config,
PageFetcher pageFetcher,
int maxBytes) |
Copyright © 2017. All rights reserved.