Module 10 — Project: an end-to-end tool-using assistant

Nine modules of pieces. This module bolts them together into the assistant that the running example promised — the one that reads a receipt, answers policy questions, remembers the trip, converts currencies, checks the ceiling and appends a spreadsheet line, all in one conversation. It is also the module where we write down the things that only matter once the whole system runs at once: error handling across tools, test coverage that catches the right bugs, and the small production concerns that decide whether next week's demo works.

The assistant, in one file

# assistant.py
from langchain_ollama import ChatOllama
from langchain_chroma import Chroma
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.output_parsers import StrOutputParser
from langchain_community.chat_message_histories import SQLChatMessageHistory
from langgraph.prebuilt import create_react_agent

from .tools import convert_currency, check_per_diem_ceiling, append_spreadsheet_row
from .policy_chain import policy_chain    # module 5
from .receipt_chain import extract_line   # module 2

# 1) The base model, shared by the policy chain and the agent
model = ChatOllama(model="qwen2.5:7b-instruct", temperature=0)

# 2) The retrieval-augmented policy chain, exposed as a tool
@tool
def ask_policy(question: str) -> str:
    """Answer a question about the internal reimbursement policy.
    Returns a plain-text answer that cites the pages used."""
    return policy_chain.invoke(question)

# 3) The agent, with all its tools
tools = [ask_policy, convert_currency, check_per_diem_ceiling,
         extract_line, append_spreadsheet_row]
agent = create_react_agent(model, tools)

# 4) Memory wrapper: one persistent history per session
def get_history(session_id: str):
    return SQLChatMessageHistory(
        session_id=session_id, connection_string="sqlite:///history.db",
    )

assistant = RunnableWithMessageHistory(
    agent, get_history,
    input_messages_key="messages",
    history_messages_key="messages",
)

Three ideas concentrated in forty lines. The retrieval chain is exposed as a tool rather than a top-level branch — the agent decides itself when to consult the policy, which is what makes conversation natural. Memory is persisted to SQLite so a restart does not lose the session. The agent's stopping rule is the default recursion_limit; we cap it below.

The four failure modes to catch

By module 10, every serious failure has a name and a specific fix.

A tool times out or errors. Wrap every tool call with a timeout and let a typed ToolException propagate. The agent will see the error, replan and often succeed on the second attempt. Track the per-tool error rate; a currency API at 1 % is normal, at 10 % it needs a fallback.

The agent loops. Cap recursion_limit at 10, log the last tool call before the cap fires, and return a graceful message rather than a stack trace. Nine of ten loops come from a single misbehaving tool; find it in the traces of module 9.

The retriever finds nothing. The policy chain must say "The policy does not cover this" — the abstention licence of module 5. Never let an empty retrieval turn into a confident guess; the assistant's credibility dies the day it invents a ceiling.

The user asks something out of scope. Add a system prompt line: "If the question is not about expense reports, decline politely and suggest the general support channel." A refusal is a feature, not a bug.

Testing the assistant

Three test tiers, each catching a different class of bugs.

Unit tests on the tools. convert_currency(-1, "EUR", "USD") raises. check_per_diem_ceiling("meal", 20, "Berlin")["over_ceiling"] is False. These run in milliseconds, they run in CI on every commit, and they catch the wrongs that are your fault, not the model's.

Integration tests on the chains. With a real model at temperature zero, invoke the receipt-extraction chain on ten fixed receipts and assert the extracted amount to two decimals. These are slow, they cost tokens, and they run once per deploy — not on every commit. Cache them.

End-to-end tests on scripted conversations. A JSON file of ten conversations of five to ten turns each, expected to end in an assertable state ("a row was appended with amount 42.30", "the assistant declined to append"). These are the tests closest to a real user, and they are what a demo actually runs on.

The evaluation set of module 9 lives alongside these; it is not a replacement. Assertion tests answer "did this specific thing happen?"; the evaluation set answers "how often does the class of things happen?". Both are needed.

Going to production: three decisions

Where the model runs. For an internal tool at low volume, an API call is fine — course 16 module 10 laid out the arithmetic. For higher volumes or on-premise data, host with vLLM behind a small internal HTTP surface. LangChain does not care which; the same ChatOllama or ChatOpenAI wraps both.

Where the data lives. Vector store, chat history, spreadsheet — three databases the assistant depends on. Give each an owner, a backup and a data-retention policy. A chat history without a TTL is a compliance issue waiting for its audit.

Who observes it. The traces of module 9 need to reach someone. A dashboard with three numbers — cost per conversation, tool-error rate per tool, evaluation-set faithfulness on the last release — is enough to catch 90 % of regressions before users report them.

The cost picture

A rough working budget for the assistant on Qwen 2.5 7B, self-hosted:

Cost driver	Order of magnitude
Model call (agent turn)	800–1 500 prompt tokens, 100–300 output tokens
Retrieval per policy question	one embedding call, one vector query
Tool calls per conversation	2 to 6
Persistent storage	negligible for a team of 50

On paid APIs at GPT-4o-mini rates, a five-turn conversation lands around one to two US cents. On a self-hosted 7B model, it is a fraction of a cent in electricity. The savings only exist above the daily traffic that fills the machine — course 16 module 10 explains the crossover.

Ship the first version behind a feature flag

The version you actually ship is version 2, once the first version has met real users. A feature flag with 10 % of the team on the new prompt for a week catches the questions your evaluation set did not cover — and there will be some. Roll to 100 % when the trace dashboard is boring.

In summary

• The expense assistant assembles in ~40 lines: model, retrieval-as-tool, tool list, create_react_agent, memory wrapper — every piece from modules 2 to 8.
• The four failure modes to catch are tool timeouts, agent loops, empty retrieval and out-of-scope questions; each has a named, specific fix.
• Test with unit tests on tools, integration tests on chains, scripted end-to-end conversations, alongside the evaluation set from module 9 — assertion tests and evaluation are complementary, not substitutes.
• Production decisions come down to where the model runs, where the data lives and who watches the traces; ship behind a feature flag and roll out when the dashboard is boring.

Next: the recap module and the final exam that turns nine modules into a certificate.