Skip to main content

Documentation Index

Fetch the complete documentation index at: https://motherfuckingsideproject.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

​
Recipes

Ready-to-use AgentXP configurations for common site types.
Pick the recipe closest to your site and adjust as needed. Each config is a complete middleware.ts you can copy directly.

​
Documentation site

Documentation sites have clean structure and work well with zero config. Add a llmsTxt page list so agents can discover your docs without crawling.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    contentSelector: 'main',
    maxTokens: 16000, // docs pages tend to be long
  },
  llmsTxt: {
    siteName: 'Acme Docs',
    description: 'Developer documentation for the Acme API',
    pages: [
      { path: '/docs', title: 'Overview', section: 'Documentation' },
      { path: '/docs/quickstart', title: 'Quick Start', section: 'Documentation' },
      { path: '/docs/api', title: 'API Reference', section: 'Reference' },
      { path: '/docs/changelog', title: 'Changelog', section: 'Reference' },
    ],
  },
})

​
SaaS landing page

Landing pages contain pricing, features, and social proof. Exclude marketing widgets and keep token limits moderate.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    excludeSelectors: [
      '#intercom-container',
      '.cookie-consent',
      '.announcement-bar',
    ],
  },
  permissions: {
    aiTrain: false,
    aiInput: true,
    search: true,
  },
  llmsTxt: {
    siteName: 'Acme',
    description: 'Project management for engineering teams',
    pages: [
      { path: '/', title: 'Home', section: 'Product' },
      { path: '/pricing', title: 'Pricing', section: 'Product' },
      { path: '/features', title: 'Features', section: 'Product' },
      { path: '/docs', title: 'Documentation', section: 'Resources' },
      { path: '/blog', title: 'Blog', section: 'Resources' },
    ],
  },
})

​
E-commerce product pages

E-commerce pages have sidebars, filters, cart widgets, and recommendation carousels. Use contentSelector to target the product content and excludeSelectors to strip shopping chrome.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    contentSelector: 'main',
    excludeSelectors: [
      '.cart-drawer',
      '.mini-cart',
      '.recently-viewed',
      '.recommended-products',
      '.cookie-consent',
      '#CybotCookiebotDialog',
      '.social-sharing',
      '.yotpo-widget',
    ],
  },
  llmsTxt: {
    siteName: 'Acme Store',
    description: 'Premium electronics and accessories',
    pages: [
      { path: '/', title: 'Home', section: 'Shop' },
      { path: '/products', title: 'All Products', section: 'Shop' },
      { path: '/deals', title: 'Deals', section: 'Shop' },
      { path: '/support', title: 'Support', section: 'Help' },
    ],
  },
})

​
Blog or content site

Blogs work well out of the box. Exclude comment sections and social widgets to keep output clean.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    contentSelector: 'article',
    excludeSelectors: [
      '.comments-section',
      '.social-sharing',
      '.newsletter-signup',
      '.related-posts',
    ],
  },
  llmsTxt: {
    siteName: 'Acme Blog',
    description: 'Engineering and product insights',
    pages: [
      { path: '/blog', title: 'All Posts', section: 'Blog' },
      { path: '/blog/getting-started', title: 'Getting Started Guide', section: 'Blog' },
      { path: '/about', title: 'About', section: 'Company' },
    ],
  },
})

​
News or media site

News sites have ad slots, trending sidebars, and comment sections inside the main content area. Aggressive excludeSelectors are required.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    contentSelector: 'article',
    excludeSelectors: [
      '.ad-slot',
      '.ad-banner',
      '.sidebar',
      '.sponsored',
      '.trending',
      '.comments-section',
      '.social-sharing',
      '.newsletter-popup',
    ],
  },
  permissions: {
    aiTrain: false,
    aiInput: true,
    search: true,
  },
})
News sites benefit the most from contentSelector: 'article'. Without it, sidebar ads and trending sections inside <main> will appear in the markdown output.

​
Dashboard or web app

App pages have toolbars, sidebar navigation, and notification drawers. Target the content area directly.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'

export default withAgentXP({
  transform: {
    contentSelector: '.dashboard-content',
    excludeSelectors: [
      '.toolbar',
      '.notification-drawer',
      '.sidebar-nav',
    ],
    maxTokens: 12000, // dashboards can have large data tables
  },
  siteHeader: {
    name: 'Acme Dashboard',
    links: [
      { text: 'Overview', href: '/dashboard' },
      { text: 'Users', href: '/dashboard/users' },
      { text: 'Revenue', href: '/dashboard/revenue' },
      { text: 'Settings', href: '/dashboard/settings' },
    ],
  },
})

​
With existing middleware

If you already have middleware (Clerk, next-intl, or custom auth), pass it as the second argument. AgentXP handles agent requests; everything else goes to your middleware unchanged.
middleware.ts
import { withAgentXP } from '@reaganhsu/agentxp-next'
import { clerkMiddleware } from '@clerk/nextjs/server'

export default withAgentXP({
  transform: {
    contentSelector: 'main',
  },
}, clerkMiddleware())
This works with any middleware that follows the Next.js middleware signature.