maxtrezzi’s jev4s connects Scala programs to TypeSafe’s Jev endpoint. Its implementation carries question types into answers and makes ordinary API failures values that callers must handle. The Spark example also shows how distributed data processing can repeat paid model calls.

Keep the question’s type through decoding

  • Scala’s Choice answer can contain a value from the application’s own enum, such as Billing or Technical.
  • Scala 3 named tuples preserve question names in the returned answers. The Scala 2.13 API uses typed keys and ordered answer tuples.
  • Literal probabilities are range-checked at compile time in Scala 3; runtime values have a checked constructor.
  • The decoder rejects missing answers, mismatched answer types, unknown choices and missing probability entries.

Those checks protect the integration contract. Typed output still needs decision-quality tests: a valid Billing value can route a ticket to the wrong team.

Spark can run the request again

A Spark Dataset describes work to perform. Separate actions can execute that work repeatedly.

The author’s local fake-transport test asserts these counts for 10 rows; the Scala test was inspected but not independently run:

  • collect() makes 10 calls, and a subsequent count() makes 10 more.
  • Sorting before collection makes 20 calls in the tested plan because Spark first samples its input.
  • Populating a cache makes 10 calls; subsequent tested reads make zero additional calls.

Persist paid answers before downstream reuse. Spark task retries can still repeat requests, so caching alone supplies no exactly-once guarantee.

Build the client where the rows run

The example constructs and shares a client inside each executor JVM, the process running Spark tasks. Its API key comes from that executor’s environment.

  • Reuse the HTTP client and its connections across tasks.
  • Retain each failed answer as an error row for selective retry.
  • Divide the planned request rate among partitions, while accounting for other jobs sharing the account.
  • Review Spark’s secret-redaction settings before passing an API key through configuration.

The partition pacer is local to the job. Concurrent jobs, client retries and repeated tasks still need account-wide controls.

Error values have a boundary

The HTTP transport converts ordinary network and response failures into errors and retries eligible failures. The client documentation explicitly allows interruption and exceptions from caller-supplied event hooks to escape. The README’s broad “No exceptions” wording needs that qualification.

The decoder checks individual probability ranges and required keys. The inspected implementation leaves probability-sum and chosen-label consistency checks to another layer.

Verification: inspected both client APIs, the Scala 3 codec and transport, and the Spark implementation and tests. The repository’s Python documentation checker passed. No Scala compilation, Spark execution, live Jev call or model-quality evaluation was performed.