Skip to main content

Documentation Index

Fetch the complete documentation index at: https://motherfuckingsideproject.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
Detection

Configure how AgentXP identifies AI agents from incoming requests.
AgentXP inspects each incoming request and determines whether it originates from an AI agent before deciding whether to serve markdown. You can configure which signals it uses and add your own bot patterns.

​
How detection works

AgentXP evaluates signals in priority order and stops at the first definitive match:
  1. accept-header — If the request includes Accept: text/markdown, AgentXP treats it as an agent with full confidence (1.0). This is a deliberate, unambiguous signal that the client expects markdown.
  2. user-agent — AgentXP matches the User-Agent string against a list of 20+ known bot patterns. A match yields confidence 0.9.
If no signal matches, the request is passed through unchanged to your Next.js application.

​
Configuration

middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  detection: {
    signals: ['accept-header', 'user-agent'],
    customAgents: [
      { name: 'MyBot', pattern: /MyCustomBot\/\d+/i },
    ],
  },
})

​
Options

​
signals

Type: ('accept-header' | 'user-agent' | 'http-signature')[] The detection methods AgentXP uses, evaluated in the order listed. Default: ['accept-header', 'user-agent']
ValueDescription
'accept-header'Match requests with Accept: text/markdown (confidence 1.0)
'user-agent'Match against known AI bot User-Agent patterns (confidence 0.9)
'http-signature'Reserved for future cryptographic agent verification
To restrict detection to only one method, pass a single-element array: 
detection: {
  signals: ['accept-header'],
}

​
customAgents

Type: { name: string; pattern: RegExp }[] Additional bot patterns to recognize beyond the built-in list. Each entry requires a human-readable name and a RegExp tested against the request’s User-Agent string. Default: [] 
detection: {
  customAgents: [
    { name: 'AcmeCrawler', pattern: /AcmeCrawler\/\d+\.\d+/i },
    { name: 'InternalAgent', pattern: /internal-agent/i },
  ],
}
Custom agents are checked after the built-in patterns. A match yields the same 0.9 confidence as other user-agent matches.

​
Built-in agent patterns

The following agents are recognized out of the box via the user-agent signal:
AgentPattern
GPTBotGPTBot
ChatGPT-UserChatGPT-User
OAI-SearchBotOAI-SearchBot
ClaudeBotClaudeBot
Anthropicanthropic-ai
PerplexityBotPerplexityBot
BytespiderBytespider
Coherecohere-ai
GoogleOtherGoogleOther
Google-ExtendedGoogle-Extended
Meta-ExternalAgentMeta-ExternalAgent
Meta-ExternalFetcherMeta-ExternalFetcher
CCBotCCBot
Applebot-ExtendedApplebot-Extended
AmazonbotAmazonbot
AI2BotAI2Bot
DiffbotDiffbot
FacebookBotFacebookBot
YouBotYouBot
Webzio-ExtendedWebzio-Extended
For the full list with regex patterns, see Supported Agents.