John Kutay, who works on growth and go-to-market engineering at Rippling, writes up tests of Jev inside GrowthOS, Rippling’s agent harness for go-to-market work. His bio notes the views are his own. The useful part is that he reports a win and a loss, and says where the line between them falls.

Assistance work versus decision work

  • He opens with a framing from Diogo Almeida, Jev’s creator at Typesafe: LLMs are almost too good to be true for human-in-the-loop assistance, and still too unreliable for many fully automated decisions. A conference slide in the article sums it up as “too good to be true” for assistance tasks and “too bad to be useful” for code-in-the-loop automation.
  • Kutay’s aim is not to remove marketers, RevOps leaders, and sellers from the loop. He wants to move them up a level, to directing agents, setting goals, and judging outcomes. That requires separating assistance work from decision work.

The win: scoring outreach

  • GrowthOS scores outreach messages on a few defined dimensions, so marketers and RevOps can see which patterns correlate with better performance. A graphic labeled as illustrative shows strategy, personalization, value proposition, and call to action, each judged independently.
  • Method: a human-labeled ground-truth set, stratified across the classification criteria and data sources, built from anonymized real-world data. Jev was compared with an equivalently prompted LLM on the same evaluation set.
  • Reported result: 90% accuracy for Jev versus 50% for the LLM baseline, and almost 8× faster.
  • The article does not name the baseline model or give the sample size, so the gap cannot be checked from outside.

The loss: judging text-to-SQL results

  • Jev “was not a universal win.” Inside a text-to-SQL agent loop, the real question is not whether a query returned rows. It is whether those rows match Rippling’s canonical definition of qualified pipeline.
  • Answering that means looking up the relevant models, filters, semantic definitions, and business context first, which an LLM can do before it decides. No figures are given for this test.

Next: retrieval without an index

  • DRY(E), “Don’t Repeat Your Embeddings”: for continuous, repeated workloads, GrowthOS builds an indexed, AI-ready representation once and reuses it across agents. It already runs semantic search over millions of sales conversations, answering questions like “What objections come up most often in Stage 1 from IT leaders?” A diagram contrasts this “precompute once, retrieve many” layer with prompting an LLM directly over warehouse text, which repeats extraction and spends tokens on every question.
  • For one-off relevance problems, he plans to skip the embedding pipeline: give the agent a bounded set of candidates and use Jev as the relevance function.
  • The idea comes from a post by Viv (@Vtrivedy10), who does applied research at LangChain Labs. For a small corpus, Jev scores every (query, document) pair and results are ranked by probability. For a large one, embeddings pull a shortlist and Jev reranks it. The diagram’s own labels note the cost: one Jev call per pair.
  • Kutay calls all of this early and experimental.

The takeaway

Evaluate Jev for repeatable, well-scoped classification and scoring. Keep the LLM for work that has to gather context and reason through it, and pair the two.

This post summarizes the full article text and its images as retrieved on September 28, 2026, including the embedded post from @Vtrivedy10. The accuracy and speed figures are Rippling’s internal results as the author reports them. The dataset and baseline are not public, and nothing here has been reproduced.