Module 3 — Data on S3 and the Feature Store

The churn dataset of course 20 has to arrive on S3 before SageMaker can train anything with it. This module lays down the prefix layout, the format choice, encryption and the Feature Store, so that every subsequent module can point at a known location.

The bucket layout

Every SageMaker project settles on one convention: one bucket per environment (dev, staging, prod), with a project-specific prefix inside. A stable layout under the prefix removes half the "where is my data?" tickets.

s3://sagemaker-eu-west-1-123456789012/
└── churn/
    ├── raw/                    # untouched CSV exported from the CRM
    │   └── 2026/09/customers.csv
    ├── processed/              # cleaned Parquet used for training
    │   ├── train/part-000.parquet
    │   ├── validation/part-000.parquet
    │   └── test/part-000.parquet
    ├── models/                 # SageMaker training output
    │   └── xgboost-2026-09-06-12-34-56/output/model.tar.gz
    ├── endpoints/              # Model Monitor captures
    ├── pipelines/              # SageMaker Pipelines executions
    └── code/                   # source distributions of training scripts

Three rules make this layout survive contact with production. Splits are on disk (train/, validation/, test/ as separate prefixes), so a training job receives them via three independent input channels and cannot accidentally leak across them. Model artifacts are named after the job, never overwritten. And code is uploaded, not embedded in the container, so a small change does not force a full image rebuild.

CSV or Parquet, choose once

SageMaker built-in algorithms accept CSV and Parquet, sometimes RecordIO. The choice is not neutral.

Aspect	CSV	Parquet
Size on disk	1×	0.1× to 0.3×
Read speed on `pandas`	slow, single-threaded	fast, columnar
Types	strings only, cast at load time	typed schema, dates preserved
Header	one line, easy to miss	metadata, no line to skip
Streaming to a training job	supported by built-ins	supported by built-ins

For the churn model, the exported CSV weighs 180 MB; the same data in Snappy-compressed Parquet weighs 22 MB. A training job on Pipe mode reads it in a fifth of the time, which shortens the whole tuning experiment of module 6. Convert once, at ingestion, and never again.

import pandas as pd

df = pd.read_csv("s3://.../churn/raw/2026/09/customers.csv")
df.to_parquet(
    "s3://.../churn/processed/train/part-000.parquet",
    compression="snappy",
    index=False,
)

Encryption, before it becomes a compliance ticket

Two settings deserve to be right from day one.

Server-side encryption with SSE-KMS and a customer-managed key. It costs nothing more per gigabyte than the default SSE-S3, and it produces a per-key audit trail in CloudTrail; every read of the bucket is logged with the identity that decrypted the object.

Block Public Access at the bucket level, and TLS-only bucket policies. A training job has no reason to reach the bucket over plain HTTP; the deny-if-not-TLS policy makes the guarantee enforceable.

The Feature Store

The layout above is enough for training. For serving, another problem appears: at prediction time you have a customer ID and need the same features the training script used, computed the same way. A homemade pipeline drifts within weeks, and every drift is a silent bug.

SageMaker Feature Store solves that with two backends sharing the same schema. The offline store is a versioned Parquet dataset on S3, used to build training sets. The online store is a low-latency key-value store (a few milliseconds per lookup), used at inference. A feature group defines the schema, the primary key (customer_id) and the event time; ingesting a record writes to both stores.

from sagemaker.feature_store.feature_group import FeatureGroup

fg = FeatureGroup(name="churn-features", sagemaker_session=session)
fg.load_feature_definitions(data_frame=df_features)
fg.create(
    s3_uri="s3://.../churn/feature-store/",
    record_identifier_name="customer_id",
    event_time_feature_name="event_time",
    role_arn=execution_role,
    enable_online_store=True,
)
fg.ingest(data_frame=df_features, max_workers=4, wait=True)

Course 33 goes deeper into feature stores as a design pattern; for this course, the useful thing to remember is the shared schema — the same feature definitions serve training set assembly and real-time lookup, which is the point.

A CSV column type is a promise you cannot keep

A CSV parsed by pd.read_csv guesses each column's type. customer_id starting with 0057 becomes an integer, event_date becomes a string, and the same file loaded on a different day may guess differently. Parquet stores the schema alongside the data; the type of every column is fixed the day you write it. This alone makes the format upgrade worth it.

Summary

One stable prefix layout under the bucket: raw/, processed/, models/, endpoints/, pipelines/, code/.
Parquet over CSV: one-fifth the size, typed schema, faster training I/O; convert once at ingestion.
SSE-KMS, Block Public Access and TLS-only policies: three checkboxes that spare you a compliance ticket later.
The Feature Store shares one schema between an offline S3 dataset for training and a low-latency online store for inference.

Next module: launch the first XGBoost training job on this data.

The bucket layout​

CSV or Parquet, choose once​

Encryption, before it becomes a compliance ticket​

The Feature Store​

Summary​

The bucket layout

CSV or Parquet, choose once

Encryption, before it becomes a compliance ticket

The Feature Store

Summary