Module 8 — Cost, latency and limits of the crew approach

Modules 1 through 7 traded a single agent for a four-agent crew. Now we look at the invoice. This module measures the number of model calls per run, the tokens each call consumes, the wall-clock latency, and the run-to-run variability that a crew inherits from a stack of stochastic decisions. Two numbers matter more than any others — cost per successful run and cost per failed run — and both need to be looked at before you commit to a crew in production.

Where the model calls go

For the running project (sequential process, four agents, five tasks, entity and short-term memory on), a healthy run produces the following calls:

Task	Agent	Model calls	Typical tokens (in / out)
extract_features	Analyst	1	1 800 / 900
draft_sections	Writer	1	2 400 / 3 200
review_draft	Reviewer	1	4 000 / 700
arbitrate	Manager	1 to 3	3 800 / 400 per call
finalise	Writer	1	4 000 / 3 500
Memory	(background)	5 embeddings	500 tokens each

Total on a clean run: 5 to 7 model calls, ~20 000 input tokens, ~9 000 output tokens, plus five embedding calls. On gpt-4o-mini for the Writer and gpt-4o for Reviewer and Manager (the mix from module 2), that lands around $0.04 per run at 2026 published prices.

Two numbers explain the spread from run to run. The Manager arbitrates one to three times depending on whether the Reviewer's verdict is approve or a reject with stylistic issues. And any tool call the Writer or Reviewer makes triggers an extra model turn — so a run where the Reviewer runs style_guide_check twice consumes two extra Reviewer turns.

Reading the meter: usage_metrics

CrewAI exposes the totals directly after kickoff():

result = crew.kickoff()
print(crew.usage_metrics)
# {'total_tokens': 27321, 'prompt_tokens': 20418, 'completion_tokens': 6903,
#  'successful_requests': 6}

That single dictionary is your finance report. Log it to the same folder as the outputs (module 3), one JSON per run, so a week later you can compute cost per successful run and cost per blocked run without replaying anything. A crew you cannot cost is a crew you cannot ship.

For per-call detail, pipe the traces to a callback. Any LangChain-compatible callback works — LangSmith, Langfuse, a home-made JSONL writer — and it will record inputs, outputs and latency per model call. Do this before you have a cost problem, not after.

Latency: sequential is not slow, agents are

The running project takes 40 to 50 seconds on a clean run. Two thirds of that is the model waiting on tokens — a Writer producing 3 500 output tokens on gpt-4o-mini is a 15-second call. One third is the framework overhead: memory retrievals, tool argument validation, the ReAct loop's own reasoning steps between tool calls.

Three levers reduce latency without changing the outputs.

• Streaming: enable it on the model that produces long outputs (Writer, finaliser). The first token arrives in about a second; the pipeline can start the next task's memory retrieval in parallel with the writer's tail.
• Parallel branches: if two tasks are independent (say, drafting sections 1–5 and sections 6–10), split them into two tasks with no context dependency and let CrewAI run them in parallel. On the running project this saves 8 to 10 seconds.
• Cheaper models on the noisy agents: an agent whose output is short and structured (the Reviewer's verdict) is a candidate for gpt-4o-mini. On the same brief, we measured 92 % agreement with gpt-4o at one third the cost.

Beyond those, you are paying for the crew's core promise (separate agents, separate turns) and cutting it further collapses the design back to a single agent.

Variability, and why it hurts more than latency

Two runs on the same brief will not produce the same draft. The Analyst may list features in a different order, the Writer may pick synonyms, the Reviewer may raise or not raise a borderline issue. On tasks where the deliverable is text, this is usually harmless. On tasks where the deliverable feeds another automated system, it is a bug.

The three levers on variability, in order of impact.

• Temperature = 0 on every agent whose output is consumed by another agent or by a machine. That is the Analyst, the Reviewer and the Manager. The Writer keeps a small temperature (0.2 to 0.3) because prose written at zero reads badly.
• output_pydantic or output_json (module 3) removes the parsing variability entirely; a validated JSON is a validated JSON, regardless of prose style.
• Fix the model version. gpt-4o today is not gpt-4o in three months. Pin gpt-4o-2024-11-20 (or whichever snapshot your provider offers) on every agent; a silent upgrade at the provider is otherwise the most infuriating regression to debug.

Hard limits of the approach

Three limits do not go away no matter how careful the wiring.

• Compound accuracy (module 1). A crew of four 95 %-accurate agents is roughly 81 % accurate end to end. On tasks where the compound accuracy is unacceptable, either shrink the crew, add a human review, or reconsider whether an agent architecture is right at all.
• The context window ceiling. Memory + context + tool descriptions + task descriptions inflates every prompt. On a ten-task run with heavy memory, the eighth task can push past 20 000 tokens; response quality drops well before the token cap.
• Cost sensitivity to a single bad decision. A Reviewer that mistakenly triggers three Manager arbitrations doubles the run cost. Cap iterations (max_iter, module 6) and log every arbitration so you notice the drift before the finance team does.

When the crew is the wrong tool

Two failure patterns should trigger a rethink rather than a tuning session.

• The cost per successful run climbs above ten times the single-agent baseline while the quality gain is under 20 %. That is a bad trade; go back to a single well-prompted agent with a human review step.
• The variability is unacceptable and cannot be pinned down. If the same brief produces very different structures across runs even at temperature 0 and pinned versions, the workflow probably belongs to a plain script that calls a model at two well-defined spots.

Budget the run before you write the crew

On a napkin, before any code: how many model calls, what token size each, what model on each. If the napkin says $0.50 per run and the business can pay $0.02, the crew was doomed before line one. This is a five-minute exercise that saves five days of tuning.

Summary

• A clean sequential run on the running project costs ~$0.04 and 40 to 50 seconds at 5 to 7 model calls; measure with crew.usage_metrics from day one.
• Cut latency with streaming, parallel independent branches, and cheaper models on structured-output agents; further cuts collapse the design back to a single agent.
• Fight variability with temperature 0 on structured outputs, output_pydantic, and pinned model versions; the Writer keeps a small temperature for readable prose.
• Hard limits: compound accuracy, context bloat, cost sensitivity to a single bad decision — budget on a napkin before you write the crew.

Next module: reading a trace when the crew talks in circles and applying the concrete fixes that unblock it.