Frontier pass rate from an open model
An open model against the frontier on τ³ banking, rung by rung, with the cost of each.
Be your own frontier.
Zeluna builds the environment your product lives in, runs the research against it, and ships what moved.
support-agent/FrontierLab is running 3 experimentsSearch⌘K
Frontier
Resolved without a reopen in 7 days, on 2,400 sealed tickets.
Score × costOver time
Resolved, serving73.8%+2.8 pts vs Opus 5
Cost per 1k tickets$6.10−85% vs Opus 5
Experiments18641 kept
Benchmark2,400sealed tickets
Your frontierKeptDiscardedFrontier APIs, as shippedLap 09
Your traces
- OpenTelemetry
- Langfuse
- LangSmith
- Braintrust
Your outcomes
- Postgres
- Snowflake
- BigQuery
- Zendesk
Your own lab,
running one loop.
Every lap starts from what your product actually did, and ends with a model that does it better.
A private benchmark of your own work. Built from your traces and graded on what happened next: the ticket reopened, the refund never filed. Not on an LLM judge's opinion.
You connect a tracing tool and the table where the outcome lands. Nothing else.
An agent that runs the experiments. Models, prompts, tools and routing, tested against your benchmark. It keeps what held, proposes it for your review, and writes up why.
A model trained on your work. When the benchmark is rich enough, it becomes the reward. A smaller open model, post-trained on your outcomes. The weights are yours.
Shipped, then measured again. Production traces flow back into the benchmark, and every new model is tested against it. Each lap starts higher.
Acme/support-agent/BenchmarkLab is running 3 experiments
Benchmark
Your tickets, graded on what happened next.
Tasks2,400from 41,230 traces
Sealed600never trained on
Graded byReopened in 7 days+ refund filed in Stripe
TicketTaskWhat happenedGrade
t_48213Refund a duplicate chargeReopened; no refund in StripeFailAn LLM judge
Pass, 9/10. “Polite, and the refund was processed.”
Your records
Fail. No refund in Stripe; the customer reopened the ticket.
t_48207Change a delivery addressShipped to the new addressPasst_48198Cancel a subscriptionCancelled; no reopen in 7 daysPasst_48190Refund outside the policy windowEscalated to a person, as policy saysPasst_48184Where is my order (split shipment)Customer wrote back: “still waiting”FailResearch
Do judges agree with what happened?
Models ranked twice on the same cases: by LLM judge and by recorded outcome.
SFT vs RL on business traces
Same traces, same reward, a recorded outcome. Which post-training method wins, and by how much?
Fidelity of synthesised environments
Executability is not correspondence. A metric for whether the synthesised world predicts production.
Questions.
What does it take to start?
Two connections: your production traces, and read access to wherever the outcome is recorded, such as a ticket reopening or a refund being filed. No labelling, and no new pipeline.
Do I need my own GPUs or infrastructure?
No. The lab runs on ours: building the benchmark, the experiments, training and serving. Your team connects the data and reviews what the lab proposes.
How is this different from an eval or fine-tuning platform?
Those start from a dataset or a judge you have already built. We start from what your product actually did, grade it on what happened next, and keep the loop running as new traffic comes in.
Who owns what comes out?
You do. The benchmark and the weights of every model we train are yours to export and run wherever you like. Or leave them with us: we store them, serve the model behind an endpoint, and keep the lab running on it.
How do I get access?
We're in a closed beta with a small number of teams whose product records an outcome to learn from. and we'll look at your use case together, and tell you plainly whether it's a fit now.