AI News — Afternoon — 2026-10-25

Better diagnosis scores can conceal worse discharge decisions

• A specialized skin-image model beats chatbots at naming lesions — This comparison shows why choosing a model requires testing the actual decision, including which dangerous cases it dismisses.
  • Dermatologist Magnus Lynch tested 200 biopsy-confirmed lesions across six diagnoses. Half required further care, while half were considered safe to discharge.
  • His specialized image network named 125 correctly, or 62.5%. GPT-6 Astra named 84 correctly, or 42%; Claude Fable 5.1 named 85, or 42.5%.
  • The network uses Inception v3, a 2015 architecture, trained further on 68,011 dermatology images. The author excluded test lesions and their patients from its training and validation data.
  • Better diagnosis accuracy did not guarantee safer discharge. At its preset threshold, the network cleared 18 of 30 melanomas; GPT-6 Astra cleared three and Claude Fable 5.1 four.
  • Changing the network’s threshold reduced missed melanomas while changing how many harmless lesions it retained. Diagnosis accuracy and discharge safety therefore need separate comparisons.
  • The download includes cases, prompts, chatbot replies and scoring code. A separate browser demonstration runs the network locally; today’s discussion surfaces an October experiment, rather than a new clinical approval.
  • Caveat: This small, biopsy-selected test uses one chatbot run per case, so its results cannot establish safety for diagnosing or dismissing real patients’ lesions. (HN 264 points · 124 comments)

Profitable deals still need checks for broken promises

• Models earn more in contract tests by breaking agreements paper notes → — Measuring an automated negotiator’s earnings alone can reward withheld payments and unfinished work that harms its counterparty.
  • An agent is a model running a multi-step task with tools. ContractSim’s October 22 revision tests Claude Opus 5, Gemini 3.6 Flash and GPT-5.6-Sol in simulated supplier relationships.
  • Across six environments, 159 of 162 negotiations reached agreement. In one high-uncertainty, low-capital environment, only three of nine model pairings produced deals benefiting both parties, averaged over three repetitions.
  • Execution was tested separately on 180 synthetic contracts, using programmed counterparties. These runs measure how models follow supplied terms, independently of their ability to negotiate them.
  • As a customer facing a cooperative supplier, GPT-5.6-Sol broke terms before the supplier did in 90% of games, often withholding the final payment. Its compliance before supplier breach still averaged 83.1% of eligible turns.
  • The base prompts explicitly maximize private payoff. Instructions against breach or warnings about future lost business reduced violations, showing that the objective and incentives affect the result.
  • Caveat: This controlled simulation lacks renegotiation and real legal enforcement, so its breach rates describe the tested prompts and game rules rather than general model trustworthiness. (Bluesky author discussion · arXiv 2608.10475)

A complete coding course for learning reasoning methods

• Raschka finishes his hands-on course on building reasoning models — The lessons let you measure extra computation and training against a small model’s answers using inspectable Python code.
  • The October 24 course page collects eight videos and about eleven hours of instruction. Today’s completion announcement links the full series, beyond the older distillation replies circulating earlier.
  • The companion repository starts from pretrained Qwen3-0.6B, a 600-million-parameter model. Python and PyTorch familiarity are recommended; the earlier language-model course is optional.
  • The lessons build a math-answer checker, then compare voting over several generated answers with scoring and revising individual answers. MATH-500 supplies 500 math problems for evaluating the changes.
  • Later lessons train from automatically checked rewards and from stronger models’ example answers. The latter method, called distillation, teaches a smaller model from a larger one’s outputs.
  • Early generation and evaluation examples can run on a processor alone. A graphics processor makes the later training practical; the setup guidance covers hardware and remote access.
  • Caveat: The course begins with an already trained model, so “from scratch” describes implementing reasoning methods rather than pretraining the underlying language model. (X @rasbt · course index and linked code)