Inference Economics on Amit Naik https://amitnaik.com/tags/inference-economics/ Recent content in Inference Economics on Amit Naik Hugo -- 0.144.2 en-us Mon, 28 Sep 2026 08:59:40 -0700 Fewer tokens per tool call can cost more overall https://amitnaik.com/til/2026-09-28-fewer-tokens-per-tool-call-can-cost-more-overall/ Mon, 28 Sep 2026 08:59:40 -0700 https://amitnaik.com/til/2026-09-28-fewer-tokens-per-tool-call-can-cost-more-overall/ GitHub describes a counterintuitive optimization failure: shorter tool outputs can lead agents to reread information and make more calls. Measure cost over a completed task, alongside task quality, rather than optimizing each individual response in isolation. Serving agents means serving repeated context https://amitnaik.com/til/2026-09-28-serving-agents-means-serving-repeated-context/ Mon, 28 Sep 2026 08:59:39 -0700 https://amitnaik.com/til/2026-09-28-serving-agents-means-serving-repeated-context/ The vLLM team's AgentX article explores serving infrastructure for multi-round agent workloads. Context reuse, parallelism, and the split between prefill and decoding become important when a task repeatedly sends overlapping context back to the model.