lib/robots/core

Functions

generateContent()

function generateContent(
   config: RobotsConfig,
   siteMapHref: string,
   logger: AstroIntegrationLogger): string

Defined in: studiocms/packages/studiocms/src/lib/robots/core.ts:172

Generates the content for the robots.txt file based on the provided configuration.

Parameters

config

RobotsConfig

The configuration object for generating the robots.txt content.

siteMapHref

string

The URL of the sitemap.

logger

AstroIntegrationLogger

The logger instance for logging messages.

Returns

string

The generated content for the robots.txt file.

Throws

Will throw an error if any policy is missing a required userAgent, if both allow and disallow entries are missing or empty, if crawlDelay is not a number or is out of the allowed range (0.1 to 60 seconds).

---

printInfo()

function printInfo(
   fileSize: number,
   executionTime: number,
   logger: AstroIntegrationLogger,
   destDir: string): void

Defined in: studiocms/packages/studiocms/src/lib/robots/core.ts:285

Prints information about the generation of the ‘robots.txt’ file.

Parameters

fileSize

number

The size of the generated ‘robots.txt’ file in KB.

executionTime

number

The time taken to generate the ‘robots.txt’ file in milliseconds.

logger

AstroIntegrationLogger

The logger instance used to log information.

destDir

string

The destination directory where the ‘robots.txt’ file is created.

Returns

void