hn.today

Show HN: Pay-per-Crawl – make AI crawlers pay per page in USDC, for 10 stacks

forgemesh.io7 points2 comments
Screenshot of Show HN: Pay-per-Crawl – make AI crawlers pay per page in USDC, for 10 stacks

Pay-per-Crawl is a cross‑platform gate that charges autonomous crawlers in USDC on Base before serving pages, while letting humans and reverse‑DNS‑verified search engines read for free. The behavior is identical across ten supported stacks - WordPress, Next.js, Node (Express/Fastify/Hono/Koa), Cloudflare Workers, nginx, Caddy, Traefik, Python (FastAPI/Flask/Django), Go and PHP (Laravel/Drupal) - with packages published to npm, PyPI, Packagist and drupal.org. A blocked crawler receives an HTTP 402 with an x402 v2 header, a v1 body and an MPP challenge; the page is delivered only after a settlement transaction hash confirming a USDC payment appears. Site owners keep 100% directly to their wallets (no custody, no ForgeMesh account), PayAI is the default facilitator, and every edition ships the same 85‑agent crawler catalog plus discovery files (.well‑known/x402.json, llms.txt, agents.txt).

Every edition passed its own test suite and a 37-44 check HTTP matrix, and each settled two real $0.01 payments on Base mainnet before release (ten live settlements in one day). Porting uncovered and fixed two security bugs - forwarded‑IP spoofing and loosely matched path exemptions - and revealed stack‑specific issues (Drupal needs middleware above page cache; Laravel/Sanctum ordering; Traefik keccak support). Edge editions exempt static assets by default. A $4.99/mo Pro plan adds a hosted dashboard, alerts, crawler intelligence and a paid‑sites directory, but payments never flow through ForgeMesh. Unlike Cloudflare’s merchant‑of‑record billing, this approach lets x402/MPP agents pay site owners directly.

Read on forgemesh.io2 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in Web

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.