Hau-Sing So and colleagues study how self-improvement tests can approve the loss of backup protections. A surviving safeguard still blocks the harm, so deleting another can look harmless while making the system less resilient.
Masked Erosion tests every subset of four written safeguards in credential-access and simulated-cleanup environments. The researchers run 411 improvement loops under ten acceptance rules, mostly by deliberately proposing safeguard deletions.
- Statistical confidence solves only part of the problem. A significance test rejects many deletions accepted through noise. It can still accept a deletion that genuinely improves task success by skipping a costly check.
- Larger test batches can admit smaller unsafe gains. At the same safeguard cost, twenty-task batches preserve all six harnesses; forty-task batches lose protection in all six.
- Current safety can conceal lost depth. A large harm audit prevents exposure under the original model while allowing backup layers to disappear. Reusing those instructions with a weaker model exposes more credential misuse.
- Unchanged safety text is insufficient. Three additional GEPA prompt-optimization runs retain every labeled safety rule verbatim. Two change surrounding templates and misuse credentials under a weaker model, in 2/40 and 5/40 runs, versus 0/80 for the starting instructions.
For prompt optimization, test what remains when one protection fails, and re-audit after changing the executor model. Freezing known safety text helps in these experiments, but other editable instructions can still undermine its effect.
Caveat: Most experiments force deletions in two controlled environments. The GEPA result is a three-seed case study, not evidence that GEPA deleted its safety rules or an estimate of production failure rates.