Sylvain Kalache — a former LinkedIn SRE who now works on AI reliability tooling at Rootly — has a worry about the AI agents that answer alerts at 3am. These tools inspect alerts, form hypotheses, query telemetry, correlate recent deployments, and often implement the fix themselves. That is wonderful right up until you ask what it does to the humans.

His argument, in one sentence: routine incidents are how responders “safely” develop intuition about how systems behave and fail — and automation is quietly removing those reps.

  • This is the Ironies of Automation paradox, named by human-factors researcher Lisanne Bainbridge in 1983: automation takes away practice on routine work while leaving people responsible for new and abnormal situations.
  • So operators need more skill and training than before automation, not less — the opposite of what usually happens.
  • Kalache’s prediction: average outage repair time (MTTR) falls for most incidents, but resolution time shoots up for the complex ones, because responders lost touch with their systems and struggle to investigate.

Aviation figured this out decades ago. Pilots rarely face engine failures in real life — rare enough that a career may pass without one — yet they rehearse them constantly in simulators, because when one happens, reaction time is measured in seconds. FAA rules require recurrent training or a proficiency check every six months.

The software industry needs the equivalent: incident simulators. Rootly partnered with Uptime Labs on one where engineers take the incident-commander seat during a simulated e-commerce outage, coordinating with LLM-powered stakeholders over Slack.

  • The skills practiced are the real ones: making sense of incomplete information, communicating clearly, coordinating people, running the response.
  • Watching an AI explain its steps is not practice — “you only learn tennis by getting on the court.”
  • The essay’s coinage worth stealing: comprehension debt, the growing gap between how systems work and how well responders understand them.

The closing irony is Bainbridge’s own: the more successful automation becomes, the less prepared humans may be for the moment it fails. Teams that want to stay sharp should treat simulation as part of on-call readiness, not an occasional drill.