Module 8 — Batch transform for large volumes

Not every prediction has to happen inside a hundred milliseconds. The marketing team of the churn use case runs the model once a month, on all four million active customers, to prioritize retention calls for the next quarter. On a real-time endpoint that scoring run costs 32 CPU-hours of invocations and needs orchestration to shard the requests. On Batch Transform it is a single job, cheaper, and it does not require an endpoint to be running the other 29 days of the month.

When to prefer batch over an endpoint

Three questions decide the choice, in order.

Do you need a prediction under one second per request? If yes, real-time. If a delay of minutes or hours is acceptable, batch is on the table.

What is the traffic pattern? A steady stream of a few requests per second, spread through the day, fits a real-time endpoint. A daily or monthly burst that scores every customer at once fits a batch job. Real-time on the burst pattern means paying 24/7 for an endpoint that idles most of the time; batch on the streaming pattern means predictions arriving hours after the input.

Do you need the input joined back to the prediction? Real-time returns one response per request and the caller knows which one it is. Batch reads a file and writes another file: preserving the customer ID next to the score is a specific option, covered below.

Launching a batch transform job

The Transformer object is the batch equivalent of the Predictor. It reuses the same Model you built in module 7.

from sagemaker.transformer import Transformer

transformer = Transformer(
    model_name=model.name,
    instance_count=4,
    instance_type="ml.m5.xlarge",
    output_path=f"s3://{bucket}/churn/predictions/2026-09/",
    strategy="MultiRecord",
    max_payload=6,
    assemble_with="Line",
    accept="text/csv",
)

transformer.transform(
    data=f"s3://{bucket}/churn/scoring-input/2026-09/",
    content_type="text/csv",
    split_type="Line",
    join_source="Input",
    wait=True,
)

Four settings deserve explanation.

instance_count=4 shards the input files across four instances in parallel; the total time drops linearly with the count, up to the number of input files. On four million customers split into 40 files of 100 000 rows, four instances finish in about 25 minutes.

strategy="MultiRecord" batches many rows into one call to the inference script. SingleRecord sends them one at a time, which is much slower and almost never what you want.

split_type="Line" tells SageMaker how to split the input files: one record per line for CSV, RecordIO for RecordIO. assemble_with="Line" does the reverse for the output.

join_source="Input" is the key to a usable result. Without it, the output file contains only the predictions in the order they came, and you must trust that order across all shards — a bug waiting to happen. With "Input", SageMaker prepends each input line to its prediction; the customer ID that was the first column of the input travels with the score.

The inference script, unchanged

The inference.py from module 7 works verbatim. predict_fn is called with a DataFrame of a hundred rows at a time (whatever max_payload fits), and returns a numpy array of scores. output_fn returns them as text; assemble_with="Line" joins them into the output file.

One consequence to internalize: the same script serves real-time and batch. That is the point of the SageMaker interface. You do not maintain two implementations, you do not risk a bug in one that the other does not have.

Cost, in numbers

Take the monthly scoring of four million customers, 40 features each.

Real-time endpoint always on: ml.m5.xlarge at $0.269 per hour, 24/7, is $194 per month. Add invocation cost of roughly $0.001 per thousand predictions — negligible here — and the scoring run itself, spread as a batch call, uses about 30 minutes of CPU-time already included in the running instance. Total: $194 per month whether you score once or every day.

Batch transform, four instances for 30 minutes: 4 × 0.5 × $0.269 = $0.54 per run, once a month.

The ratio is not close. Batch wins whenever the traffic pattern allows it, by two orders of magnitude on this workload. The real-time endpoint remains right when the marketing team wants a single customer scored within a hundred milliseconds of a support call — and that is a different requirement from monthly scoring.

Joining the output back

The output S3 object, one line per input:

CUST-00187,0.041
CUST-00188,0.612
CUST-00189,0.018

Two pandas lines to merge it with the rest of the customer file:

scores = pd.read_csv(
    "s3://.../churn/predictions/2026-09/scoring-input.csv.out",
    header=None, names=["customer_id", "churn_probability"],
)
customers = pd.read_parquet("s3://.../churn/customers/2026-09/")
result = customers.merge(scores, on="customer_id", how="left")

The left join catches any customer for which a score is missing — a data-quality signal the batch operator should raise.

When a batch job fails

Two failure modes dominate.

A malformed input row (a comma inside a text field, a missing column) stops the whole job on some framework containers, and only the affected shard on others. Validate the input file before launching; a schema check on ten sampled rows costs a second and saves a re-run.

A model that hits an unseen feature value (a new product code the trained encoder does not know) fails silently on the affected rows and produces NaNs in the output. Module 10 covers Model Monitor's role in surfacing this class of failure automatically.

Choose the size once, then never again

ml.m5.xlarge with instance_count=4 and strategy="MultiRecord" is a good default for scoring one to ten million rows on a small model. Save it in a script; the temptation to re-tune the instance type before every run is the second-largest source of on-call fatigue in ML operations, after re-tuning hyperparameters before every training.

Summary

• Batch replaces an endpoint whenever prediction latency in minutes is acceptable and traffic is clumpy; the monthly scoring is the canonical case.
• The Transformer reuses the Model of module 7; MultiRecord batches rows, join_source="Input" glues predictions back to identifiers.
• Cost drops by two orders of magnitude on the running example: $0.54 per run versus $194 per month for an always-on endpoint.
• Validate the input schema before launching; a single malformed row can stop a shard and cost the run.

Next module: chain training, evaluation and deployment together with SageMaker Pipelines.