Skip to main content

Module 2 — Models, prompts and output parsers

Module 1 decided that the assistant needs the framework. This module puts down the three atomic pieces every chain uses: a chat model, a prompt template, and — the one that saves days of debugging — an output parser that turns model text into a data structure your code can consume without a regular expression.

The chat model interface​

Every model in LangChain implements the same contract, called Runnable. For chat models the input is a list of messages and the output is one message.

from langchain_ollama import ChatOllama
from langchain_openai import ChatOpenAI

model = ChatOllama(model="qwen2.5:7b-instruct", temperature=0)
# or, identical downstream:
# model = ChatOpenAI(model="gpt-4o-mini", temperature=0)

reply = model.invoke("Extract the total from: 'Dinner Berlin, 42.30 EUR'.")
print(reply.content)

Swapping providers is one import. That single fact is why teams write their prototype against Ollama and their production call against a paid API without rewriting a chain.

The full contract exposes four methods you use daily. invoke returns one reply. stream yields tokens as they arrive. batch sends a list of inputs in parallel — cheaper and much faster than a loop. ainvoke, astream, abatch are the async twins for a web server.

Prompt templates​

Hard-coding a prompt inside a function is fine until the day you want to A/B two variants, translate it or reuse it across chains. ChatPromptTemplate treats the prompt as a small program with typed variables.

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You extract the total amount from a receipt line. Reply with a number and a three-letter currency code, nothing else."),
    ("human", "{line}"),
])

chain = prompt | model
chain.invoke({"line": "Dinner Berlin, 42.30 EUR"})
# -> AIMessage(content='42.30 EUR')

Two rules save the most time. Keep templates in one file so a wording change is a single diff. And never pass user text through the template without escaping curly braces — f"..." and PromptTemplate both interpret them, which is a real source of silent bugs when a user pastes JSON.

Output parsers: from free text to a data structure​

The expense assistant needs {"amount": 42.30, "currency": "EUR"}, not the string "42.30 EUR". An output parser is what does that conversion.

The clean way in 2026 is with_structured_output, which asks the provider to return validated JSON matching a Pydantic schema — natively, when the model supports it.

from pydantic import BaseModel, Field

class ReceiptLine(BaseModel):
    amount: float = Field(description="Total, without currency symbol")
    currency: str = Field(description="Three-letter ISO code, e.g. EUR")

structured = model.with_structured_output(ReceiptLine)
structured.invoke("Dinner Berlin, 42.30 EUR")
# -> ReceiptLine(amount=42.3, currency='EUR')

The value of this is not the syntax. It is what happens when the model returns something ill-formed: with_structured_output raises a typed exception you can catch, retry with a stricter prompt or route to a fallback. A bare chain.invoke returning a stray "about forty-two euros" would flow into your database as a string and blow up two days later.

When structured output is not available​

Some open models do not implement native JSON schema. LangChain gives you two fallbacks.

PydanticOutputParser embeds a JSON schema in the prompt as {format_instructions} and parses the response.

from langchain_core.output_parsers import PydanticOutputParser

parser = ReceiptLine and PydanticOutputParser(pydantic_object=ReceiptLine)
prompt2 = ChatPromptTemplate.from_messages([
    ("system", "Extract the total. {format_instructions}"),
    ("human", "{line}"),
]).partial(format_instructions=parser.get_format_instructions())

chain2 = prompt2 | model | parser
chain2.invoke({"line": "Dinner Berlin, 42.30 EUR"})

JsonOutputParser is the same idea without Pydantic — a plain dict comes back — for when validation is done downstream.

The failure that hurts most: parse errors​

A production chain will meet parse errors. The model returns an extra sentence, a code fence, a trailing comma, or the wrong field name. Three habits contain the damage.

Fail loud, not silent. Wrap the parser in a try-except that logs the raw output and returns a typed error to the caller. Never return None and let the next step guess.

Retry with the error as context. OutputFixingParser wraps a parser: on failure it makes a second call including the parse error and the original text, and re-parses. It costs one round-trip and rescues most malformed replies. Cap the retries at one — a model that fails twice will fail ten times.

Constrain generation at the source. Temperature zero, an explicit system prompt saying "no prose, JSON only", and a small max_tokens cut the parse-failure rate more than any downstream repair.

The number-vs-string trap

A model will happily return "42.30" instead of 42.30. Pydantic will coerce it, but a hand-written dict[str, float] will not, and your total + tip will concatenate strings. Trust the type, and let the parser enforce it.

In summary​

  • Every chat model in LangChain implements the same Runnable interface (invoke, stream, batch), which is why swapping providers costs one import.
  • ChatPromptTemplate keeps prompts editable and typed; escape user text that may contain { to avoid silent template substitution bugs.
  • with_structured_output with a Pydantic schema is the modern path from free text to typed data; use PydanticOutputParser as a fallback for models without native JSON schema.
  • Handle parse errors as first-class events: log raw output, retry once with OutputFixingParser, set temperature zero and instruct the model to emit JSON only.

Next module: composing these atoms into chains with LCEL, and running them in parallel, in a branch or in a stream.