Gen AILab
Blog

I built my first voice agent in a weekend

What broke, what surprised me, and why latency matters more than the model.

Sahil Sharma

Sahil Sharma

Published on Gen AI Lab, 30 October 2026. 4 min read

I thought the hard part would be the model. It was silence. Two seconds of quiet on a call feels like the line went dead.

What made it work

  • Stream every step, so the first word comes fast.

  • Let the caller interrupt at any moment.

  • Hand off to a person the moment it is unsure.

Start small, and test it on a real call the same day.

Related writing