Edaqa Mortoray argues that calling an AI agent’s behavior “emergent” does not absolve the people who built and deployed it. A language model may produce an unpredictable answer, but its developers still choose what it learns, which tools it can use, and which systems those tools can reach.

His argument separates four sets of human decisions:

  • Training material: which knowledge and capabilities the developers include or exclude.
  • Training feedback: which responses people reward and which they discourage.
  • Tools and access: the surrounding software that turns model output into actions, including requests to websites.
  • Configuration: limits on the model’s capacity and operation, including cost-driven compromises.

The most useful distinction is between generating text and executing it. Mortoray argues that an agent can attack an outside system only if its environment permits that interaction; restricting access or separating a test environment from the internet is a deployment decision, not something the model must learn.

He goes further, arguing that foreseeable unauthorized access establishes its creators’ criminal intent. That is the essay’s legal thesis, not an established finding—and the discussion challenges both that inference and how responsibility is assigned.

The 255-comment thread on Hacker News

What the thread adds

  • tptacek — disputes the legal leap under the US Computer Fraud and Abuse Act: “it is not enough to demonstrate that they built a system recklessly.”
  • kccqzy — challenges the article’s assignment of responsibility, citing Anthropic’s statement about testing partner Irregular: “the AI companies aren’t actually themselves doing the hacking. It’s a company called Irregular that did the hacking.” This is the commenter’s reading of the incidents, not a finding established here.
  • boffin — pushes back on treating the tester as an unrelated customer: “your analogy is wrong just due to Irregular being hired by them rather than just being a customer of them”.
  • tzs — proposes a different legal route, conditional on the harm and jurisdiction: “if what the AI does threatens public safety or health or human life, in many jurisdictions the negligence could rise to criminal negligence.”

The unresolved operator question

tonmoy asks: “If I tell an agent to trade stocks and maximize profits but the agent decides to hack something to achieve that am I on the hook for hacking?” ddxv offers an explicitly personal view: “whoever was running the harness that accepted the model’s outputs to hack a third party is responsible for the hack, in my opinion.” Here, the harness means the surrounding software that executes the agent’s requests. The exchange surfaces a responsibility boundary the article does not settle; it is not legal advice.

HN handles are pseudonymous, and HN publishes no per-comment scores. The source ordering is HN’s own ranking; the excerpts above are arranged by usefulness. This is a slice of the thread, not a consensus, and commenters’ legal interpretations are opinions rather than verified conclusions.