When OpenAI’s GPT-6 Astra launch demo showed the model working on a circuit board in KiCad, the EEBench team — the people behind the hardware-design language atopile — asked the question that usually gets skipped: how do you actually measure whether the electronics an AI produces are any good?
Their answer is a benchmark that grades circuits the way a compiler and test suite grade code — except the tests measure voltages. Models work on circuits written as declarative code rather than clicking around a graphical CAD tool, and each submitted design gets built, simulated, and checked against spec limits using real manufacturer parts with their datasheet tolerances.
That last part is where the real world bites. A task like “keep the processor alive for 20 ms when power drops” seems simple — add a capacitor — until a real ceramic capacitor delivers far less than its advertised capacitance once voltage is across it, parts have tolerances, and extra capacitance costs money, space, and slows recovery. The benchmark pushes every component to worst-case tolerance corners and grades electrical performance against cost.
September leaderboard results:
- Claude Opus 5: 61.6%
- Grok 4.6: 57.1%
- Claude Fable 5.1: 56.4%
- GPT-5.5: 42.3% — OpenAI’s tested models sit further down the table, and GPT-6 Astra hasn’t run yet
- xAI published EEBench in Grok 4.6’s model card — a sign frontier labs now treat electronics ability as a category worth reporting
The same simulation harness doubles as a training signal: a failed run shows which voltage missed its limit and which corner failed, which gives post-training a reward richer than “the schematic looks plausible.”
The verdict is measured: for a useful and growing set of circuit problems, yes — but nobody should ask it to design a pacemaker and blindly install the result. Trade-offs between electrical performance, cost, and supply are the actual substance of engineering, and that’s where the headroom is.