Latency

What is Latency

Latency is how long it takes to get a usable response back, and in an agent loop it is not just the model thinking, it is every tool call inside that turn too: a file read, a search, a command execution, each adding real wall-clock time before the final answer is ready. A turn that makes five sequential tool calls is five times slower than a turn that needed only one.

This is one of the practical reasons subagents and parallel dispatch matter: running several independent lookups at once, rather than one after another, can cut a task's total latency substantially even though the total amount of work is the same. A harness that only ever does things one step at a time pays for every step in sequence, whether or not the steps actually depended on each other.

Latency is a genuine tradeoff, not just a number to minimize. A faster, more clipped response can mean less verification, fewer tool calls to double-check a claim, less thorough exploration before answering. Whether low latency or high thoroughness matters more depends entirely on the task, a quick lookup and a careful audit have very different right answers here.