Skip to content
Cloudflare API
Start here
API Reference

AI Audit

AI AuditRobots

Get robots.txt rules
client.aiAudit.robots.get(RobotGetParams { zone_id, subdomain } params, RequestOptionsoptions?): RobotGetResponse { userAgents, sitemaps, status }
GET/zones/{zone_id}/ai-audit/robots
Bulk get robots.txt rules
client.aiAudit.robots.bulkGet(RobotBulkGetParams { zone_id, body } params, RequestOptionsoptions?): RobotBulkGetResponse { userAgents, sitemaps, status }
POST/zones/{zone_id}/ai-audit/robots/bulk
ModelsExpand Collapse
RobotGetResponse { userAgents, sitemaps, status }

Parsed robots.txt rules for a single domain.

userAgents: Record<string, UserAgents>

Map of user-agent string to its parsed rules.

allow: Array<string>

List of allowed path patterns.

disallow: Array<string>

List of disallowed path patterns.

contentSignals?: ContentSignals { ai-input, ai-train, search }

Content signal directives from robots.txt.

"ai-input"?: "yes" | "no"

Whether AI input usage is permitted.

One of the following:
"yes"
"no"
"ai-train"?: "yes" | "no"

Whether AI training is permitted.

One of the following:
"yes"
"no"
One of the following:
crawlDelay?: number

Crawl delay in seconds.

sitemaps?: Array<string>

List of sitemap URLs found in robots.txt.

status?: number

HTTP status code from fetching the robots.txt file.

RobotBulkGetResponse = Record<string, item>

Map of hostname to parsed robots.txt rules.

userAgents: Record<string, UserAgents>

Map of user-agent string to its parsed rules.

allow: Array<string>

List of allowed path patterns.

disallow: Array<string>

List of disallowed path patterns.

contentSignals?: ContentSignals { ai-input, ai-train, search }

Content signal directives from robots.txt.

"ai-input"?: "yes" | "no"

Whether AI input usage is permitted.

One of the following:
"yes"
"no"
"ai-train"?: "yes" | "no"

Whether AI training is permitted.

One of the following:
"yes"
"no"
One of the following:
crawlDelay?: number

Crawl delay in seconds.

sitemaps?: Array<string>

List of sitemap URLs found in robots.txt.

status?: number

HTTP status code from fetching the robots.txt file.