hn.today

Making My Blog Readable by AI Agents

deallocatedobjects.com36 points9 comments
Screenshot of Making My Blog Readable by AI Agents

Jake Marsh describes adapting his static blog so AI agents and search engines can retrieve and interpret its content more easily, using generated files and nginx configuration rather than adding a framework. Each post is available as Markdown through a .md URL, and clients requesting text/markdown receive that format from the regular URL through content negotiation. The setup marks responses with Vary: Accept to prevent caches mixing formats; Markdown responses also identify the HTML canonical URL and carry a noindex directive to avoid duplicate indexing. The broader argument is that these improvements are inexpensive and especially useful for a blog whose author wants its work to be found and cited.

The site also adds /llms.txt, which explains the blog and links to every post’s Markdown version, plus a single file containing all posts and a Markdown homepage. Its robots.txt explicitly permits major crawlers and signals that search, AI input, and AI training are allowed - a choice the author contrasts with what a product site might make. Atom and JSON feeds now provide summaries and Markdown links. Metadata was expanded with descriptions, canonical URLs, social preview tags, and structured data; summaries are generated from rendered text to avoid malformed old descriptions. The update also removes a mobile zoom restriction, gives each post a proper preview image, and notifies Bing and related services through IndexNow when new posts are published.

Read on deallocatedobjects.com9 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.