Jinlin Liu, Wei Chen and Xiaojin Zhang test what happens when an agent’s skill library grows. They keep tasks and relevant skills fixed, then add unrelated instructions. Agents increasingly stop opening the skills they need.
Their main study uses 82 tasks, three models and three attempts per condition. Programs check completed work against task requirements.
- Adding 1,000 unrelated skills reduces relevant-skill loading from 73.6% to 27.4%. Runs opening no skills rise from 9.3% to 51.6%.
- Task success within three attempts falls from 51.6% to 38.6%. Models sometimes solve tasks without the supplied skills, masking the discovery failure.
- A separate model call choosing skills from the same full library retains 73.9% selection recall. After that filtering, the executor opens 62.0% of relevant skills and reaches 51.6% task success.
- With 10,000 distractors, searching for candidates before model selection achieves 50.4% task success, versus 41.5% for similarity search alone.
Masking irrelevant descriptions largely restores loading in a smaller experiment. Asking the agent to plan its skill use helps less than separating selection from execution. The authors interpret this as competing instructions, beyond a simple shortage of context space.
Track which skills are offered, selected and actually opened alongside final task results. A successful answer can hide a broken procedure library.
Caveat: Preselection can miss skills needed later and adds delay. The released code omits raw experiment logs and third-party skill contents.