Pirzada Suhail and colleagues train a small language model to work backward from an answer to a possible prompt. Their PTP method learns from reversed sequences of tokens, the text fragments a model processes. It then receives additional training on real prompts paired with model-generated answers.
The authors report these results on a 100-prompt ShareGPT test set:
- Same-model recovery: the reported exact-match score is 64.77% for Qwen3-0.6B Chat, versus 25.44% for the strongest Output-to-Prompt baseline in that comparison.
- Different models: the score falls to 17% for Llama-2 7B Chat and 11.36% for GPT-4o outputs.
- Different training data: using 30,000 Instructions-2M examples instead of the 400 ShareGPT training prompts yields 11.03% on the ShareGPT test set.
The recovered prompt can also be fed back into the original model to check whether it produces a similar answer. That measures a different capability from recovering the exact historical input: many prompts can produce similar responses.
Caveat: “data-free” describes synthetic pretraining only; the main result also uses 400 real prompts and assumes access to the target tokenizer. These are author-reported results from a narrow test, not a general method for proving which prompt produced an arbitrary text.
The appendix supports the value of synthetic pretraining over training on those 400 prompts alone. It also estimates roughly 38 million generated tokens per pretraining epoch. A single answer is enough at reconstruction time, but building the inverse model requires substantial prior querying.