Rohan Pandey of DigitalOcean demonstrates an optimization agent on AI Engineer; the video also credits Balaji Varadarajan.

The search loop

  • Optuna proposes serving configurations across repeated trials, scores each run and uses the results to choose subsequent candidates.
  • The starting recipe covers short chat, long chat, retrieval-augmented generation and batch classification workloads; token lengths can be changed.
  • The operator supplies a study name, starting YAML, hardware and trial count; the run emits an HTML report and an optimized YAML recipe.

Results and scope

  • Pandey reports 17.5% more generated tokens for a GPT-OSS example; its report shows 50 trials, while the command demo specifies 24.
  • A separate nine-trial example reportedly doubled throughput and nearly halved inter-token latency. These are presenter-reported runs, not independent benchmarks.
  • The description promises additional production studies, but this recording instead contains a short tuning demo; it does not demonstrate those claims.

“Reduce the time it takes to manually find the best configs.”