loading
Back to the blog

[ Thesis ]

Narration engineering: real-time speech for the AI sales engineer inside your product

Context engineering decides what a model reads. We had to engineer what an agent says out loud while it works — every spoken line is a claim about a screen that keeps changing.

rtrvr.ai Team·July 24, 2026·8 min read

Narration engineering: real-time speech for the AI sales engineer inside your product

Rover is the AI sales engineer inside your product. One script tag: it gives every visitor a live demo of the real product and walks every new customer through onboarding and setup. Getting it to reliably operate real software was the hard problem we expected, and it's why Rover is #1 on the Halluminate Web Bench at 81.4%.

A sales engineer talks while they drive. So does Rover — and at first its voice described step two while the screen already showed step five. Everything it said was true when the model wrote it. By the time you heard it, it wasn't.

It took three production incidents to work out why.

The hands are already moving. Is what I was about to say still true?

No chatbot has to ask that. It has nothing to fall behind. Three things we got wrong before we understood it:

  • A queue is the wrong shape for speech. Buffer narration and the agent describes step two while the screen already shows step five.
  • The browser will not let you be proactive. A local "unlocked" flag is a lie that the first hard refresh exposes.
  • Finishing is a delivery problem, not a rendering one. Our own cleanup timer was truncating the most important sentence of the session.

Everyone else is solving a different problem

Every voice agent answers the same question: has the human finished speaking? That problem is close to solved. LiveKit ships a 135M-parameter model whose entire job is predicting whether your sentence is finished. ElevenLabs treats interruption, eagerness, and delivery-critical speech as separate controls. Production guidance says audio must stop within about 60ms of a barge-in or the agent feels deaf. All of it is built around one boundary: the moment a human stops talking.

Rover has no such boundary. It is acting continuously inside a real product, and its own actions keep invalidating the sentence it was about to say. That isn't conversation. It's commentary, and commentary fails differently: an agent that talks over you is rude, an agent that falls behind is wrong.

Voice AI solved conversation. We had to solve commentary.

Where we got stuck

Actions are fast. Speech is slow. A click takes 200ms; describing it takes four seconds. So a run that touches six things in fifteen seconds produces an agent calmly describing step two while the screen shows step five. It sounds like it isn't watching the same screen you are.

The instinct is to speed the voice up or trim the backlog. Both are wrong.

Speech is not a queue of strings. It's a set of claims, each with a shelf life.

Once "the repository is connected" stops being the newest true thing, saying it out loud tells the visitor nothing. It just fills the air.

The life of one spoken claim: the model authors it inside the same tool call, an identity check drops duplicates and stale runs, five bounded lanes hold at most five live claims, one sentence is spoken to its natural end, and a receipt records what completed so nothing replays

What we rebuilt

The model authors meaning. The platform owns delivery. Everything in between is what we've started calling narration engineering — the same discipline context engineering brought to what a model reads, applied to what an agent says out loud.

The claim is written inside the same tool call the model was already making, so there is no second round trip and no narration-only prompt. Everything after that is ours:

  • Five bounded lanes, not a queue. One active sentence, one carry-forward claim, the newest progress claim, one question barrier, one protected result. Five things can be alive at once whether the task runs ten seconds or ten minutes.
  • Carry-forward with an expiry. Exactly one unspoken claim survives a topic change, and only for six seconds. After that it belongs in the transcript, not the speaker.
  • Identity, not string matching. The same sentence about a genuinely new thing still gets said. The same event arriving twice through two transports gets said once.
  • A question ends the turn. Rover stops after asking. It never queues a status line behind its own question.

Three scars from production

The page unloads mid-sentence. Audio cannot survive a document unload. Continuity can. We journal which sentences completed, carry that receipt through the navigation, and resume at the next sentence boundary — never mid-word. A completed claim can never play again. That sounds like bookkeeping, but it's what stops Rover from starting the tour over every time the page changes.

The browser refuses to let you be proactive. Chrome will not autoplay audio for a visitor with no history on your site, and play() is asynchronous authorization, not a request — a resolved promise is the only proof sound actually happened. So Rover attempts once, honestly, and when it's blocked it shows one clear control instead of pretending.

Finishing is a delivery problem. We had an eight-second timer cleaning up after each run. Real closing statements run longer than eight seconds. So the cleanup was truncating the single most important thing Rover says all session — the result — and every dashboard said the run succeeded. Server completion, UI rendering, and audio playback are three independent lifecycles. Only the visitor gets to end the last one.

That bug has a smaller cousin worth naming, because no log will ever show it to you. The browser speech fallback splits long text at 180 characters and speaks each piece as a separate utterance. The engine restart between them lands as an audible seam in the middle of your closing sentence. You can only find it by listening.

What we measure

The metric that matters is claims delivered exactly once, while still true.

So the telemetry counts claim states — accepted, started, completed, carried, superseded, expired — not audio events. Alongside it runs an authorship rate: what share of spoken lines the model actually wrote, versus grounded fallback copy. A rise in fallback is never a voice problem. It's a prompt problem, and this is where it surfaces first.

That opens three questions we couldn't ask before: where people interrupt (pacing data, per playbook), which claims are always stale by the time they're spoken (a prompt edit the data can propose itself), and whether the second visit sounds like it remembers the first.

What you can take from this

If you're building an agent that does anything while it speaks, four things transfer:

  • Give every spoken line an identity and an expiry. Deduplicate on the claim, not the characters. A queue will happily narrate a world that no longer exists.
  • Treat playback authorization as real. play() resolving is the only proof that sound happened. Anything else is a guess.
  • Separate the lifecycles. Server completion, UI rendering, and audio are three different clocks. Cleaning up on the wrong one silently eats your best sentence.
  • Listen to it. The seam, the stale line, the two-second dead air — none of it appears in a dashboard. We found our worst voice bug by wearing headphones.

What this is actually for

The category Rover competes with is the interactive product tour, and Navattic's 2026 benchmark of ~5,000 B2B SaaS sites is honest about its ceiling: 18% of sites now run a demo CTA, and even top-decile demos finish at 40% on about three minutes of attention. Good numbers, for a thing that asks a stranger to click through a replica of your product.

Rover isn't a replica and doesn't ask for clicks. It runs the real product, on the visitor's own account, and talks them through it — the same capability that completed 77% of 121 multi-step tasks in our own logs. The voice isn't decoration on that. It's the reason someone watches an agent work instead of taking the mouse back.

Frequently asked questions

Does the voice slow the agent down?

No. No browser action waits for speech. Navigation durability is what makes that safe: the agent acts immediately because the claim outlives the page it was authored on.

What happens if the browser blocks audio?

Rover attempts once and, if blocked, shows a single control to turn voice on. Text, choices, and the run itself continue silently. Microphone access is separate and only requested when someone presses the microphone.

Is this a voice bot bolted onto a chat widget?

The opposite. The spoken claim is produced by the same model call that decides the next action, so the words and the hands come from one decision. That is the only reason they stay in sync. The buy-vs-build math on rebuilding that coupling in-house is not close.


Most of this work turned out to be deciding what not to say.

Every software company has three or four people who can just do it for the customer — the solutions engineer who builds it live on the call, the CSM who screen-shares and finishes the setup. They are the highest-leverage people you have, and they are rationed to your biggest accounts.

Rover is that person on every account. One script tag: a live demo for every visitor, then real onboarding and setup work inside your app. It doesn't wait to be asked — it reaches out first, recommends the next step, and does it with them.

If you're lifting PLG activation, covering more accounts without adding headcount, or giving mid-market the attention only your top 10% gets, that's the job.

Get on the pilot program

More from the blog

On this page

  • Everyone else is solving a different problem
  • Where we got stuck
  • What we rebuilt
  • Three scars from production
  • What we measure
  • What you can take from this
  • What this is actually for
  • Frequently asked questions
  • Does the voice slow the agent down?
  • What happens if the browser blocks audio?
  • Is this a voice bot bolted onto a chat widget?