hn.today

Previous-Token Prediction Based LLM Inversion for Prompt Reconstruction

arxiv.org47 points10 comments
Screenshot of Previous-Token Prediction Based LLM Inversion for Prompt Reconstruction

PTP tackles the problem of recovering a prompt from an LLM’s response, where many different prompts can produce similar text. Earlier approaches treat recovery as a search for a semantically plausible prompt and may depend on external training data, fine-tuning, or access to model weights and token probabilities. PTP instead uses the target model as a black box: it generates training examples synthetically from that model, then trains a separate inverse language model from scratch. Rather than predicting the next token as a standard LLM does, the inverse model predicts previous tokens, learning to generate prompts conditioned on observed responses.

The method is designed to recover prompts that reproduce the response, not merely prompts with a similar meaning. Sampling from the inverse model can yield several candidate prompts, which need not match the original wording but should induce similar outputs when passed to the target model. The authors report that PTP generalizes across datasets and can reconstruct prompts for responses produced by other LLMs, suggesting some transfer beyond the model used to generate its training data. On token-based measures of both prompt and response reconstruction, they report better results than prior methods. The work’s central contribution is a practical, black-box inversion strategy that frames prompt reconstruction as a learned generative process.

Read on arxiv.org10 comments on Hacker News

Summary generated by AI from the linked article. hn.today is not affiliated with Hacker News or Y Combinator.

More in AI

Mapping every neuron in Qwen3-8B

Mapping every neuron in Qwen3-8B

Researchers found individual neurons in Qwen3-8B that reliably steer its replies into distinct personas or styles, including pirate, German-speaking, and childlike voices. Nudging these neurons changed outputs across many questions without prompting or fine-tuning, revealing how specific behaviors can be localized and edited inside a language model. (fermionresearch.com)

ChatGPT for SNES

ChatGPT for SNES

ChatGPT for SNES runs a 2.2-million-parameter language model on the Super Nintendo’s Super FX 2 chip, with an optional Wikipedia fact book. The unofficial experiment can run in an emulator or on a flash-cart-equipped console, but its tiny model often gives incorrect answers. (bxfoundry.com)

The daily digest

Today's best Hacker News stories, summarized and screenshotted, one email a day.