Core API

Full API reference for @reaganhsu/agentxp-core and @reaganhsu/agentxp-next exports.

@reaganhsu/agentxp-next
@reaganhsu/agentxp-core

`withAgentXP(config?)`

Next.js middleware factory. Wrap your middleware export with withAgentXP to intercept agent requests and serve optimized markdown automatically.

// middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP()

Returns a Next.js middleware function compatible with the middleware.ts convention.

Config

All fields are optional.

interface AgentXPMiddlewareConfig {
  transform?: {
    contentSelector?: string
    excludeSelectors?: string[]
    handlers?: Record<string, (el: Element) => string>
    maxTokens?: number
  }
  permissions?: {
    aiTrain?: boolean
    aiInput?: boolean
    search?: boolean
  }
  baseUrl?: string
  llmsTxt?: {
    siteName?: string
    description?: string
    pages?: LlmsTxtPage[]
  }
  siteHeader?: SiteHeaderConfig
  overridesDir?: string
}

`transform.contentSelector`

Type: stringCSS selector for the main content area. When set, AgentXP extracts only the matching element. Auto-detected from main, article, [role="main"], #content, or .content if omitted.

`transform.excludeSelectors`

Type: string[]Additional CSS selectors for elements to strip before conversion (e.g., cookie banners, chat widgets).

`transform.maxTokens`

Type: numberMaximum token target for the markdown output. Content is truncated by section when exceeded. Defaults to 8000.

`permissions.aiTrain`

Type: booleanPermit AI providers to use content for model training. Reflected in the Content-Signal response header. Defaults to false.

`permissions.aiInput`

Type: booleanPermit AI agents to use content as inference input. Defaults to true.

`permissions.search`

Type: booleanPermit search indexing. Defaults to true.

`baseUrl`

Type: stringBase URL for resolving relative links in the markdown output. Auto-detected from the incoming request when omitted.

`llmsTxt.siteName`

Type: stringSite name written to the # Heading of /llms.txt. Defaults to "Site".

`llmsTxt.description`

Type: stringSite description written as the > blockquote in /llms.txt.

`llmsTxt.pages`

Type: LlmsTxtPage[]Explicit list of pages to include in /llms.txt. See the LlmsTxtPage type under @reaganhsu/agentxp-core.

`siteHeader`

Type: SiteHeaderConfigManual site navigation header prepended to every markdown response. When omitted, AgentXP auto-extracts navigation from the page’s <header> and <nav> elements.

`overridesDir`

Type: stringPath (relative to project root) to a directory of hand-written .md files that take priority over auto-generated markdown. Requires Node.js runtime — use createMarkdownHandler instead if you are on Edge.

`AgentExperience`

React component. Add it to your root layout to inject agent-optimized metadata with no visible UI.

import { AgentExperience } from '@reaganhsu/agentxp-next'

export default function RootLayout({ children }) {
  return (
    <html>
      <body>
        {children}
        <AgentExperience siteName="My Site" description="Product docs" />
      </body>
    </html>
  )
}

The component renders no visible output. It injects:

A JSON-LD WebSite structured data block
A <link rel="alternate" type="text/markdown" href="/llms.txt" /> tag
A <meta name="robots"> tag with noai / noimageai directives when training or inference is disabled

Props

`siteName`

Type: stringSite name written into the JSON-LD WebSite schema. Optional.

`description`

Type: stringSite description written into the JSON-LD WebSite schema. Optional.

`aiTrain`

Type: booleanWhen false, adds the noai directive to the robots meta tag. Defaults to false.

`aiInput`

Type: booleanWhen false, adds the noimageai directive to the robots meta tag. Defaults to true.

`createMarkdownHandler(config)`

Import from @reaganhsu/agentxp-next/route-handler. Creates a Next.js Route Handler GET function that serves markdown with file-based override support. Runs in Node.js runtime (not Edge), so it can read files from disk.

// app/api/md/[[...path]]/route.ts
import { createMarkdownHandler } from '@reaganhsu/agentxp-next/route-handler'

export const GET = createMarkdownHandler({
  overridesDir: 'content/markdown',
  siteHeader: { name: 'My Site', links: [{ text: 'Docs', href: '/docs' }] },
})

Config

interface RouteHandlerConfig {
  overridesDir: string
  baseUrl?: string
  siteHeader?: SiteHeaderConfig
  bypassHeader?: string
}

`overridesDir`

Type: string (required)Directory for hand-written markdown overrides, relative to process.cwd(). A request to /pricing will check <overridesDir>/pricing.md first. The root path maps to index.md.

`baseUrl`

Type: stringBase URL for resolving relative links. Defaults to request.nextUrl.origin.

`siteHeader`

Type: SiteHeaderConfigManual site navigation header prepended to every response. When omitted, AgentXP auto-extracts from the page’s HTML.

`bypassHeader`

Type: stringInternal header name used to fetch page HTML without triggering the middleware again. Defaults to "x-agentxp-internal".

`detectAgent(request)`

Detect whether an incoming request is from an AI agent.

import { detectAgent } from '@reaganhsu/agentxp-core'

const result = detectAgent(request)
if (result.isAgent) {
  console.log(result.agent?.name) // e.g. "ClaudeBot"
}

Parameters

`request`

Type: Request (required)A standard Web API Request object. Inspects the Accept and User-Agent headers.

Return type

interface DetectionResult {
  isAgent: boolean
  agent?: AgentInfo
}

interface AgentInfo {
  name: string
  confidence: number
  method: 'accept-header' | 'user-agent' | 'http-signature' | 'behavioral'
}

isAgent (boolean, required): true when the request is identified as coming from an AI agent.
agent (AgentInfo): Present when isAgent is true.
- name (string, required): Matched agent name (e.g., "ClaudeBot", "GPTBot"), or "unknown" when detected via Accept: text/markdown with no matching User-Agent.
- confidence (number, required): Detection confidence score from 0 to 1. 1.0 for accept-header detection; 0.9 for user-agent pattern match.
- method (string, required): The signal that triggered detection. One of 'accept-header' | 'user-agent' | 'http-signature' | 'behavioral'.

`transformForAgent(html, options?)`

Full HTML-to-markdown transformation pipeline. Processes a raw HTML string through four stages: expand hidden content, extract main content, convert to markdown, clean and finalize.

import { transformForAgent } from '@reaganhsu/agentxp-core'

const result = transformForAgent(html, { baseUrl: 'https://example.com' })
console.log(result.markdown)
console.log(result.tokenEstimate)

Parameters

`html`

Type: string (required)Raw HTML string to transform.

`options`

Type: TransformOptionsOptional transform configuration.

contentSelector (string): CSS selector for the main content area. Auto-detected when omitted.
excludeSelectors (string[]): CSS selectors for elements to remove before conversion.
baseUrl (string): Base URL for resolving relative links to absolute URLs in the output.
maxTokens (number): Maximum token target. Content is truncated by section when exceeded.
headingStyle (‘atx’ | ‘setext’): Markdown heading style. 'atx' uses # prefixes (default); 'setext' uses underlines.
title (string): Page title override. Auto-extracted from the HTML when omitted.
description (string): Page description written into the output metadata.
includeNav (boolean | SiteHeaderConfig): Prepend a site navigation preamble to the output. Pass true to auto-extract from the page’s <header>/<nav>; pass a SiteHeaderConfig object to use a manual override. Omit or pass false for no preamble.

Return type

interface TransformResult {
  markdown: string
  tokenEstimate: number
  metadata: {
    title: string
    sourceUrl?: string
    description?: string
  }
  processingTimeMs: number
}

markdown (string, required): The agent-optimized markdown output.
tokenEstimate (number, required): Estimated token count of the markdown output, calculated as Math.ceil(markdown.length / 4).
metadata (object, required): Extracted page metadata.
- title (string, required): Page title extracted from the HTML or provided via options.title.
- sourceUrl (string): Source URL, set from options.baseUrl when provided.
- description (string): Page description extracted from the HTML or provided via options.description.
processingTimeMs (number, required): Wall-clock time in milliseconds for the full pipeline.

`generateLlmsTxt(options)`

Generate a standards-compliant llms.txt file following the llmstxt.org specification.

import { generateLlmsTxt } from '@reaganhsu/agentxp-core'

const content = generateLlmsTxt({
  siteName: 'My App',
  description: 'Developer documentation for My App.',
  baseUrl: 'https://example.com',
  pages: [
    { path: '/docs', title: 'Documentation', section: 'Documentation' },
    { path: '/pricing', title: 'Pricing', section: 'Product' },
  ],
})

Parameters

`options`

Type: LlmsTxtOptions (required)

siteName (string, required): Site name written as the top-level # Heading of the file.
description (string): Site description written as a > blockquote below the heading.
baseUrl (string, required): Base URL prepended to each page path to form absolute links.
pages (LlmsTxtPage[], required): List of pages to include. Each entry:

interface LlmsTxtPage {
  path: string        // URL path, e.g. "/pricing"
  title: string       // Page title
  description?: string
  section?: string    // Section grouping, e.g. "Documentation"
}

maxPages (number): Maximum number of pages to include. Defaults to 15. Pages beyond this limit are silently omitted.

Return type

Returns a string containing the complete llms.txt content.

`defineConfig(config)`

Type-safe configuration factory for agentxp.config.ts. Returns the config object unchanged — its only purpose is to provide TypeScript type inference and editor autocompletion.

// agentxp.config.ts
import { defineConfig } from '@reaganhsu/agentxp-core'

export default defineConfig({
  permissions: {
    aiTrain: false,
    aiInput: true,
    search: true,
  },
  transform: {
    maxTokens: 6000,
  },
})

Parameters

`config`

Type: AgentXPConfig (required)Your AgentXP configuration object. All fields are optional.

interface AgentXPConfig {
  detection?: {
    signals?: ('accept-header' | 'user-agent' | 'http-signature')[]
    customAgents?: { name: string; pattern: RegExp }[]
  }
  transform?: {
    contentSelector?: string
    excludeSelectors?: string[]
    handlers?: Record<string, (el: Element) => string>
    maxTokens?: number
  }
  llmsTxt?: {
    autoGenerate?: boolean
    include?: string[]
    exclude?: string[]
    description?: string
    maxPages?: number
  }
  analytics?: {
    enabled?: boolean
    endpoint?: string
  }
  permissions?: {
    aiTrain?: boolean
    aiInput?: boolean
    search?: boolean
  }
}

Return type

Returns the config typed as AgentXPConfig. Use with a default export in agentxp.config.ts.

defineConfig is a zero-overhead identity function — it exists purely for TypeScript autocompletion. When using Edge Runtime middleware, pass your config directly to withAgentXP() rather than loading it from a separate file, since dynamic import() and process.cwd() are not available in the Edge Runtime.

For full configuration documentation, see Configuration overview.

​Core API

​withAgentXP(config?)

​Config

​transform.contentSelector

​transform.excludeSelectors

​transform.maxTokens

​permissions.aiTrain

​permissions.aiInput

​permissions.search

​baseUrl

​llmsTxt.siteName

​llmsTxt.description

​llmsTxt.pages

​siteHeader

​overridesDir

​AgentExperience

​Props

​siteName

​description

​aiTrain

​aiInput

​createMarkdownHandler(config)

​Config

​overridesDir

​baseUrl

​siteHeader

​bypassHeader

​detectAgent(request)

​Parameters

​request

​Return type

​transformForAgent(html, options?)

​Parameters

​html

​options

​Return type

​generateLlmsTxt(options)

​Parameters

​options

​Return type

​defineConfig(config)

​Parameters

​config

​Return type

Core API

`withAgentXP(config?)`

Config

`transform.contentSelector`

`transform.excludeSelectors`

`transform.maxTokens`

`permissions.aiTrain`

`permissions.aiInput`

`permissions.search`

`baseUrl`

`llmsTxt.siteName`

`llmsTxt.description`

`llmsTxt.pages`

`siteHeader`

`overridesDir`

`AgentExperience`

Props

`siteName`

`description`

`aiTrain`

`aiInput`

`createMarkdownHandler(config)`

Config

`overridesDir`

`baseUrl`

`siteHeader`

`bypassHeader`

`detectAgent(request)`

Parameters

`request`

Return type

`transformForAgent(html, options?)`

Parameters

`html`

`options`

Return type

`generateLlmsTxt(options)`

Parameters

`options`

Return type

`defineConfig(config)`

Parameters

`config`

Return type