Agent Systems: Resource-Aware Optimization

Figure

Figure: a policy routes work among cheap, targeted, deferred, and stopped execution paths.

In one sentence. Match model and effort to the value of the task, treating cost and latency as design constraints rather than as things measured afterwards.

Problem

Agent systems consume model calls, context, retrieval, tools, worker slots, and time. Maximizing answer quality without measuring cost and latency is not a production design.

Intent and structure

task difficulty and value → resource policy → method selection → measured result

The main trade-off is among quality, latency, and cost, with reliability and reproducibility as additional constraints.

Practical policies

• Use deterministic code for exact computation.
• Use small models for classification, extraction, or formatting.
• Reserve stronger reasoning or deep search for high-value uncertainty.
• Reduce repeated work with caching, context pruning, batching, and early stop.
• Use asynchronous execution for long-running work and checkpoints for resume.

Degradation testing

Measure what happens when model quality, retrieval quality, or tool availability decreases. A graceful fallback should expose lower confidence or reduced scope, not silently claim the original quality.

Real-world application

A high-volume support service can send routine requests to a cheaper path, reserve stronger reasoning for ambiguous cases, and defer work that is not urgent. The policy must expose quality limits and escalation behavior so cost savings do not silently change the service promise.

Reference basis

This is Gulli, Agentic Design Patterns, Chapter 16, “Resource-Aware Optimization,” combined with Mitra, System Design for the LLM Era, Chapter 2, which treats latency, cost, model routing, and dynamic traffic control as architectural concerns.