Module 2 — System prompt and role framing

Module 1 rewrote the running case as a single user message. It works, but every call re-sends the same framing, and any change to the framing has to be edited in dozens of places at once. The chat completion API offers a cleaner separation: system for what is stable, user for what varies. This module explains what that separation actually buys you, and what it does not.

Two roles, two lifetimes

A chat completion accepts messages with roles. The three that matter here are:

Role	Contains	Changes across calls
system	durable framing: task, constraints, format	almost never
user	the specific input to process	every call
assistant	past model answers, if replayed	reused for few-shot in module 3

Placing the framing of the running case in system makes it explicit that it should not depend on which email comes in:

from openai import OpenAI

client = OpenAI()

SYSTEM = """You extract four fields from customer complaint emails:
reason, product, urgency, requested_action.

Rules:
- Use only information present in the email.
- If a field is not mentioned, write "not stated".
- Do not add commentary.
- Return four labelled lines in this order: reason, product,
  urgency, requested_action.
"""

def extract(email: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": SYSTEM},
            {"role": "user", "content": email},
        ],
    )
    return response.choices[0].message.content

The user message is now exactly the email, nothing else. This matters for cost, for logging, and for the injection defence of module 8.

Instruction priority is not absolute

A tempting mental model is: the system message is a rule, the user message is data, therefore the system always wins. That is wrong, and repeating it in production has burned enough teams to be worth correcting immediately.

Modern models are trained to give system messages higher priority, not absolute priority. When a user message contradicts the system with enough insistence — or when the user message contains text that mimics a system instruction — the model can flip. This is the mechanism behind prompt injection and is treated in detail in module 8.

The practical rule that follows: the system message is where you put what should be stable, not what must be safe. Safety-critical constraints require additional mechanisms outside the prompt itself, such as output validation and tool-level permissions.

Persona: useful versus decorative

"You are an expert customer support analyst with fifteen years of experience" is the kind of opening most tutorials suggest. On measurable tasks like our field extraction, this decorative persona changes outputs by a fraction of a percent. Do not confuse fluency of the prompt with quality of the output.

A useful persona, by contrast, restricts the model's behaviour in a way the constraints cannot express directly:

SYSTEM = """You are a strict field extractor.
You never rephrase the email. You never add greetings or apologies.
You only output the four requested fields.
"""

The value here is not the word "expert" but "strict", "never" and "only" — words that narrow the acceptable output space. As a rule of thumb, a persona is useful when removing it visibly changes the output on ten test emails; if not, it is decoration.

Stability across calls

One benefit of putting the framing in the system message is reproducibility. To measure it, run the same prompt ten times on the same input with a fixed decoding parameter:

outputs = [extract(email) for _ in range(10)]
distinct = len({o.strip() for o in outputs})
print(f"{distinct} distinct outputs out of 10")

With a well-framed system message and a low temperature (see below), distinct should be one or two. If it is closer to ten, the framing is weak and you are reading a different task each time.

The temperature parameter controls randomness. On an extraction task, the right value is close to zero:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[...],
    temperature=0,
)

At $T = 0$ the model picks the most likely token at each step, and outputs are nearly deterministic. Reserve higher temperatures ($T$ around $0.7$ to $1.0$) for creative writing, where you want variety.

Match temperature to task

Fact extraction, classification, JSON generation: $T = 0$ or $0.1$. Brainstorming, drafting variants, generating training data: $T$ around $0.7$. A single global default is almost always wrong for at least one of your tasks.

When the system message is ignored

Two situations cause the system framing to have little effect, and both are worth recognising:

• The user message is much longer than the system. If the system is fifty tokens and the user is five thousand, the framing gets diluted. Move truly critical constraints to the very end of the user message as well, or use a shorter, sharper system.
• The framing contradicts the input. "Return English only" on a system prompt followed by a French email still sometimes yields a French answer, because the input evidence is stronger. Fix with an explicit second reminder placed after the input.

A stable system prompt is a versioned artifact

The system message is code, not free text. Store it in a file, review diffs, and never edit it in place on a production run. Module 10 turns this into a full library discipline.

In summary

• The chat API separates system (durable framing) from user (per-call input); putting the running case's rules in system cleans up the pipeline and prepares injection defence.
• Instruction priority is higher, not absolute: the system message is where you put what should be stable, not what must be safe.
• A useful persona restricts behaviour with words like "strict" or "never"; a decorative "expert with fifteen years of experience" changes outputs by a fraction of a percent.
• Low temperature ($T \approx 0$) is right for extraction and classification; keep higher temperatures for tasks that need variety.

Next module: examples inside the prompt, and how many are enough.