edu.uci.ics.crawler4j.robotstxt

Class RobotstxtParser

java.lang.Object

edu.uci.ics.crawler4j.robotstxt.RobotstxtParser

public class RobotstxtParser
extends Object

Author:

Yasser Ganjisaffar

Constructor Summary

Constructors

Constructor and Description
RobotstxtParser()

Method Summary

Modifier and Type	Method and Description
static HostDirectives	parse(String content, RobotstxtConfig config)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RobotstxtParser

public RobotstxtParser()

Method Detail

parse

public static HostDirectives parse(String content,
                                   RobotstxtConfig config)