Atul, cofounder of Filed, describes a review harness for the conventions that tests and linters miss. Jev inspired the approach, but Filed actually uses Perplexity’s pplx-decider in production, including its image input for design checks. This is an implementation report, not a Jev benchmark.

One rule file, two readers

  • Nested rules.yaml files hold a specific mistake or compliance statement, applicable paths, a suggested fix, a reference file, and labeled examples.
  • Before editing, the coding agent asks the CLI for relevant rules; after editing, the reviewer judges the change against the same wording.
  • Mistake probabilities are inverted so every displayed score points toward compliance; low scores become hints, not automatic merge vetoes.
  • The agent fixes a hint or records a dismissal reason. Atul’s described loop hands off to a human after three review rounds.

Test the wording—not just the generated code

Atul reports that “reuses existing helpers” gave both a broken change and its fix 0.97. Naming the actual duplicate-tool mechanism separated them at 0.17 versus 0.82. Those are author-reported examples, not independently reproduced measurements or a held-out accuracy result.

  • Each rule needs violating and compliant cases, including innocent edits to the same files.
  • Rule tests measure the gap between the weakest compliant case and strongest violation; stored hashes detect changes that need retesting.
  • Screenshot pairs alter one design issue at a time. False alarms and misses become new cases—but tuning on those cases can overfit.

Cheap model input is not cheap review overall

Atul reports a large review’s model-input charge at roughly $0.04, while another full review took roughly a minute. Neither figure was reproduced here; the input charge excludes screenshot capture, harness maintenance, agent retries, and human attention. His observed usage suggests the state is billed again for each question, making narrow rule scope important.

The conclusion is deliberately conservative: prevent mistakes with architecture first, then use deterministic tests or lint, then a model, and finally prose guidance. Provider failures are recorded as “review did not run”, not a clean review. Code and screenshots leave the machine, so the report excludes taxpayer fixtures and production payloads.

Typed output is not a correct decision. Atul explicitly treats a score as “look here,” not a calibrated measurement of code quality. Human review still owns the judgment.