On a live sales call, a good answer is only useful if it arrives while the customer is still on the line. Agaton is a deep voice AI platform that analyzes every customer conversation (why we built it), and part of what we do is help sales and customer service agents (both human and AI) capture revenue opportunities quickly in the moment.
We recently put NVIDIA Nemotron 3.5 Lightning to work on exactly this. By routing the right knowledge questions to it, we got answers to our customers' teams faster. Here is what we built, what we measured, and why an open-weight model was a good choice.
When the answer has to arrive mid-call
Agaton is used across the revenue organization, from agents on live calls to CEOs and leaders who rely on it to identify and capture revenue opportunities that are currently missed. For the agents in the conversation, speed is everything. A customer asks about a price, a subscription, a routine or a product detail, and the answer has to arrive while they are still on the line. In that setting, latency is part of the product. A good answer that arrives a few seconds late is, in practice, a missed answer.
For a leading global telecom customer, we wanted to find out whether an open model could take on part of our knowledge retrieval workload and respond faster. NVIDIA Nemotron 3.5 Lightning was the model we chose to test.
Sending each question to the right model
Not every question needs the same model. Agaton has a large history of what users ask and which tool calls follow, and we used it to train a classifier that recognizes the knowledge retrieval questions Nemotron answers well. It routes based on the questions users have asked before, and on the questions most likely to call our knowledge tool. Those questions go to Nemotron for faster delivery. Everything else continues to go to the closed model we already use.
We host Nemotron 3.5 Lightning ourselves through Hugging Face, running it with vLLM on a GPU in our own cluster. It has mixture-of-experts architecture with 30 billion parameters in total and approximately 3 billion active parameters per token. Because only a subset of its parameters is active for each token, it can offer a favorable compute and latency profile compared with a similarly sized dense model, which suits a workload where every second counts.
The first words arrive twice as fast
We measured speed in two ways. The first is time to first token (TTFT): how long it takes before the user sees the first words appear in streaming mode. The second is end-to-end latency: the full time it takes to deliver a complete response.
On the questions we routed to Nemotron 3.5 Lightning, average TTFT dropped by about half, so the first words reach the agent about twice as fast. End-to-end latency fell by about 20 percent. For a live call, TTFT is especially important because it indicates when the agent first receives useful feedback. We also track end-to-end latency and answer quality because an early first token is not sufficient on its own.
To test this, we ran more than 70 real questions from users at that customer through both models, five times each. An LLM judge checked that every answer had the facts it needed and called the right tools.
"On a live sales call, speed and latency performance is key. By routing the right knowledge questions to NVIDIA Nemotron 3.5 Lightning, we cut time to first token by about half."
John Kristensen, co-founder and COO, Agaton
Why an open model fits this use case
Speed was the first reason to try Nemotron, but not the only one.
The first is room to keep improving. Because the model provides downloadable weights, we can adapt it to Agaton's needs through further post-training. With our large corpus of sales conversations, we can fine-tune an open model to get even better at Agaton's tasks, and we own the resulting model.
The second is control of data. Sales conversations contain sensitive information. We already redact personal data actively, and hosting our own open model adds another layer of governance and security for the companies we work with.
Hearing how it is said with Audio Flamingo 3
Speed is only half of what makes a sales conversation work. The other half is listening to how something is said. A hesitation does not always mean no. Confidence, energy and tone all carry meaning that a transcript alone loses.
That is why we also include NVIDIA Audio Flamingo 3 among the models behind Agaton's prosodic analysis. It helps us read signals such as hesitation, confidence and energy, and find hidden revenue in calls: missed buying signals, missed churn signals, and openings for upsell and cross-sell. There is more work ahead here, and we will share more as it develops.
What comes next
This is a first step. As open models keep pushing the boundary, we plan to try more of them and keep learning from real usage. Our next step is to post-train Nemotron among other open models on Agaton's own dataset of sales conversations and knowledge artifacts.
For a platform used live, where every second counts, choosing the right model for specific tasks matters as much as the model itself. Nemotron 3.5 Lightning gave us a fast, open option for the questions it handles best, and let us keep our customers' sales agents one step ahead in the conversation. To see how Agaton opens the black box of sales, visit agaton.ai.
