Jessica Hullman argues that AI-assisted science depends on people who can defend, question and revise its findings. A correct answer matters. So does having someone who can explain what it rests on when a result fails.
- Checks have boundaries. Her hypothetical AI-discovered treatment passes a trial, but could still exploit a weakness in the trial itself.
- Authorship carries obligations. A named author who cannot explain a paper’s claims leaves reviewers with nobody able to answer substantive criticism.
- Corrections need continuity. An agent can retract a claim in one session without carrying that correction into the next.
- Judgment needs practice. Even an accountable AI would need people capable of questioning its evidence and recognizing evasive answers.
Hullman connects this to how scientific communities choose worthwhile questions and build understanding through criticism. Caveat: this is a philosophical argument about AI and scientific practice, not a study measuring whether AI use erodes researchers’ skills.
The 103-comment thread on Hacker News adds a proposed workflow and disagreement about what verification already provides.
What the thread adds
- colijobles — “First, autonomous or human-directed AI agents submit their work to a public repository.” Then: “This work of interpretation, synthesis and question formation should be recognised and rewarded in peer-reviewed academic journals.”
- txrx0000 — “But the AI didn’t give me an answer directly, it gave me a proof that I can run through a verifier. And I could figure out how the verifier works. That is a partial understanding.”
- RugnirViking — “LLMs are capable of amazing feats of reverse engineering but are terrible at explaining and abstracting things well.” They add: “they could be trained to be better at this if we focused on it”.
HN handles are pseudonymous; no per-comment scores are public. The fetched ordering is HN’s ranking; this is a slice of the thread, not a consensus.