Brave Sells Its Search Index to AI, and Your Website Is Probably in It

Brave’s Search API licenses access to its web index to AI firms including Cohere, Mistral, Perplexity, and You.com. The New York Times alleges in a lawsuit that Brave copied more than 30 billion webpages without clearly identifying its crawler or giving website owners a way to block it, then supplied indexed material to AI companies. Brave has not responded to the suit. The central technical issue is that Brave says its crawler does not use a distinct user-agent name and follows Googlebot’s access rules instead. That makes it difficult for a site to permit Google while excluding Brave: blocking Brave specifically is not possible through robots.txt, while using a noindex tag can also remove a page from Google.

The writer found 16 pages from his own site in Brave’s results, including content and a description that had already been changed, indicating that the index stores copies rather than simply retrieving live pages. He argues that ordinary search can send readers to publishers, whereas AI access may deliver page excerpts and metadata directly to a model, which can generate answers without sending users back. The Times and several other newspapers have filed lawsuits, alleging copyright violations and other misconduct; earlier disputes have also focused on whether Brave’s crawler can be detected or blocked. A court’s decision could shape how publishers’ material is indexed and licensed for AI, though a ruling about one publisher would not automatically settle the rights of all sites.