Uses of Class
org.attribyte.api.http.util.RobotsTxt

Packages that use RobotsTxt

Package	Description
org.attribyte.api.http.util

Uses of RobotsTxt in org.attribyte.api.http.util

Fields in org.attribyte.api.http.util declared as RobotsTxt

Modifier and Type	Field and Description
static RobotsTxt	RobotsTxt.NO_ROBOTS

Methods in org.attribyte.api.http.util that return RobotsTxt

Modifier and Type	Method and Description
static RobotsTxt	RobotsTxt.parse(java.lang.String host, Client httpClient, java.lang.String userAgent, java.util.Set<java.lang.String> preserveAgents, Logger logger) Creates a robots.txt from the standard location (/robots.txt).