Module 2 — Compute targets: instances, clusters, serverless compute

The workspace from Module 1 has no CPU of its own. Every job — training, batch scoring, pipeline step — runs on a compute target you pick per job. Azure ML offers four kinds, and mixing them up is the single most common source of surprise bills.

The four kinds of compute

Compute instance — a single VM permanently attached to your identity, meant for interactive work: notebooks, VS Code Remote, ad-hoc exploration. It is the closest thing to a personal dev box in the cloud.

Compute cluster — an auto-scaling pool of identical VMs, used by submitted jobs. Nodes spin up on demand and go back to zero when idle. This is where the demand-forecast training will run.

Serverless compute — Azure ML picks the VM size from a resources block in your job spec; you never provision a cluster. Convenient for one-off jobs, less predictable when a team runs dozens per day.

Attached compute — an external resource (an AKS cluster, a Synapse Spark pool, an HDInsight cluster) registered with the workspace. Useful when the team already runs the shared infrastructure; out of scope for our forecast project.

Right-sizing a cluster for the forecast training

The training job in Module 5 will read two years of weekly sales for roughly 200 stores and 5 000 SKUs, fit a gradient-boosted regressor, and finish in under fifteen minutes. That fits comfortably on a mid-range CPU VM. The cluster is created once, upfront:

az ml compute create --type AmlCompute \
  --name cpu-cluster-forecast \
  --size Standard_DS3_v2 \
  --min-instances 0 \
  --max-instances 4 \
  --idle-time-before-scale-down 300 \
  --tier Dedicated

Three parameters carry the whole design.

--min-instances 0 is the single most important knob. It lets the cluster scale to zero when idle, so you pay nothing during the six days a week when no job runs. Setting it to one because "spin-up is slow" turns a €30/month cluster into a €600/month one, silently.

--max-instances 4 caps parallelism. A hyperparameter sweep (Module 5) will fan out four trials in parallel; without a cap, an accidental sweep of a hundred trials would launch a hundred nodes and burn your monthly quota in an afternoon.

--idle-time-before-scale-down 300 waits five minutes after the last job before releasing nodes. Long enough to reuse the same node for a sequence of related jobs; short enough to matter on the bill.

Serverless: when it is the right pick

Serverless compute skips cluster creation altogether — you request the machine size inside the job:

# job.yml
command: python train.py --data ${{inputs.sales}}
environment: azureml:forecast-env@latest
resources:
  instance_type: Standard_DS3_v2
  instance_count: 1
inputs:
  sales:
    type: uri_folder
    path: azureml:sales-2y:1

For a scientist running one-off exploration, this saves the round trip through cluster provisioning. For a team, prefer named clusters: cost dashboards can attribute usage to a cluster by name and tag, whereas serverless charges appear pooled under the workspace.

Low-priority (Spot) nodes and their honest trade-off

Setting --tier LowPriority gives Azure the right to evict a node at any moment, in exchange for a discount that typically runs 60 – 90 %. For a fault-tolerant workload — a hyperparameter sweep where losing one trial is fine, or a batch scoring job that re-processes missed rows — the trade is excellent. For a single, long, non-resumable training run, an eviction thirty minutes in costs the entire compute time already spent. The rule of thumb: use low-priority for parallel or checkpointed workloads, dedicated for the critical path.

Compute instance ≠ compute cluster

Newcomers habitually run training in a notebook on a compute instance. It works, and it teaches nothing. When the notebook is closed, the instance keeps running and keeps billing. When the training takes six hours, the notebook loses its kernel connection and the run is lost. When two scientists both need a GPU, the instance sits on a fixed one and cannot share.

The right split: compute instance for the notebook that submits the job, compute cluster for the job itself. The pattern that Module 5 will use throughout.

Region quotas: the ceiling you did not know you had

Every subscription has a per-region quota for cores, broken down by VM family (Standard_DS, Standard_NC for GPU, low-priority separately). A cluster whose max-instances × cores-per-VM exceeds the quota simply cannot scale up: jobs queue forever with no error message.

Check yours before the first big sweep:

az ml compute list-usage --location westeurope -o table

The output shows current versus limit per family. A quota increase is a ticket in the Azure Portal that typically resolves in a few hours; requesting it before the deadline, not the night before, is a form of professional courtesy to future you.

The auto-scale-down trap

A cluster with --min-instances 1 and --idle-time-before-scale-down 30 sounds economical. It is not: any pause between jobs longer than thirty seconds keeps a node alive continuously. Either scale to zero and accept the spin-up delay, or accept a permanent floor. There is no thrifty middle setting.

Summary

• Four compute types: instance for interactive work, cluster for submitted jobs, serverless for one-offs, attached for shared infrastructure.
• The critical cluster knob is --min-instances 0; it is what makes idle time free.
• Low-priority discounts 60 – 90 %, but only for eviction-tolerant workloads like sweeps and batch scoring.
• Region cores quotas are a hard ceiling; check them before the first big sweep, request increases early.

Next module: data assets and datastores — how the forecast training reads two years of sales from Blob Storage without hard-coding a URL.