NGA is a research project exploring Jev-style decision models and the systems needed to use their judgments in business workflows. Its distinction: consistent answers, safe retries and traceable actions are separate problems.

I inspected the repository’s README, baseline scorer and reporting code, and experiment records at commit 1fb4ae7. I did not run its models or reproduce its results. The proposed agent framework and workbench are not finished products.

Test decisions, not just their format

  • The research plan pairs equivalent cases with relevant changes. A model should preserve a decision when irrelevant details change and respond when the rule or evidence changes.
  • The baseline scorer compares a Hugging Face decoder’s candidate-label token probabilities. It also supplies empty-input, question-only, constant and uniform controls to expose shortcuts.
  • The reporting code fits thresholds on training splits and reports results by held-out group, source and question type. A typed answer can still be wrong; consistent wrong answers are not a success.
  • The maintainers say their initial baseline needs an erratum because batch-composition noise affects its results. It is not evidence of a native typed-readout advantage or a passed consistency gate.

Keep decisions separate from effects

  • The proposed framework would bind tasks and versions, adopt decisions, control external actions, recover durable waits and replay recorded inputs without live effects.
  • Reusing a decision on retry does not prove that similar cases receive the same answer. Preventing a duplicate payment or other action also depends on the framework and external system.
  • The planned Java SDK, openJiuwen adapter and workbench remain proposals. The repository says its integration and research gates are not yet established.

The project publishes reference evaluations of existing open models, but those are the maintainers’ results, not a demonstrated improvement from NGA training. Much of the underlying data, weights and full run output lives outside Git.