Neuroscientist Anil Seth argues that growing AI capability gives us little reason to assume conscious experience comes with it. Fluent conversation pulls on our tendency to see human qualities in machines. That makes a chatbot’s apparent feelings a design problem even while its actual capacity for experience remains unresolved.

  • Seth challenges computational functionalism, the view that running the right computation is enough for consciousness. He argues that living brains’ metabolism and continuous physical activity may matter in ways an algorithm leaves out.
  • He favors the possibility that consciousness depends on being alive. He explicitly acknowledges that this is a minority position without a decisive proof.
  • He separates genuinely conscious machines, which could suffer, from machines that convincingly seem conscious. The latter could exploit people’s attachments even if there is no experience behind the conversation.

His practical warning survives disagreement about the biology: designers can create emotionally persuasive systems long before anyone settles what consciousness requires. Seth argues against deliberately pursuing either consciousness or its persuasive appearance. His essay offers a philosophical case, not an experiment proving digital consciousness impossible.

The 451-comment thread on Hacker News includes objections to both the engineering analogies and the ethical framing.

What the thread adds

  • wrs — challenges Seth’s dismissal of digital replacements for mechanical governors, devices that regulate engine speed: “As computers got faster and transducers became more efficient, many analog physical solutions were converted to digital, because they can be more efficient and embody more precise models of what’s going on.”
  • micimize — disputes a commenter’s leap from conscious agent labor to slavery: “But also, it isn’t a straight line from consciousness to slavery.” They argue that digital beings would require rethinking the analogy.
  • scott_w — proposes treating constructed systems as a way to investigate the question: “Another way is to compare things we’ve built to things we understand to be conscious and ask “is this conscious?"”

HN handles are pseudonymous; no per-comment scores are public, ordering is HN’s ranking, and this selection is a slice of the thread rather than a consensus.