Lesson 6 — Recap and frequent questions
The five lessons in five sentences
- Cloud AI reduces to four categories — raw compute, managed APIs, machine learning platforms and hosted foundation models — and you should work down that list rather than up, because building a custom model for a task a managed API handles is the most common expensive mistake here.
- The three platforms provide the same seven capabilities and differ mainly in coherence and in which data platform they pair with, so the criterion that should decide is where your data already lives.
- Bills are dominated by idle accelerators, peak-provisioned serving, accumulated storage and egress rather than by training compute, and the largest single lever is interruptible spot capacity.
- Answering whether data can go to an AI service takes seven specific questions, residency binds on processing as well as storage, and compliance requirements for high-risk uses converge almost exactly with good MLOps practice.
- Data gravity is the only heavy lock-in, cheap portability at the container and interface boundaries is worth buying, and genuine switchability is worth investing in only for hosted model APIs.
The decision path
A cost checklist to run before provisioning
| Action | Takes |
|---|---|
| Budget alerts at 50, 80 and 100 percent | 5 minutes |
| Tag every resource with project and owner | 10 minutes |
| Enable auto-shutdown on notebook instances | 5 minutes |
| Storage lifecycle rules with an agreed retention | 15 minutes |
| Per-team quotas so a mistake is bounded | 20 minutes |
| Estimate peak serving cost, not average | 30 minutes |
| Check whether batch would do instead of real time | 15 minutes |
Under two hours, and it prevents the majority of escalated cloud bills.
Twelve frequent questions
1. Which cloud should I choose?
The one holding your data. Moving data is slow and expensive, cross-cloud operations cost effort permanently, and the platform feature gaps close within a year or two. Override this only for a hard residency requirement, a capability that exists nowhere else, or contract economics that genuinely change the picture.
2. Do I need an ML platform at all?
Not for one model. A container on a virtual machine, or a scheduled batch job, is a complete production deployment for many use cases. Platforms earn their cost with multiple models, multiple people, or a need for managed monitoring and governance.
3. Managed API or my own model?
Managed for anything general — transcription, translation, document text extraction, standard image labels. Your own when the task is specific to your domain, when volume makes per-request pricing painful, or when data cannot leave your infrastructure. Test the managed option on fifty real examples before deciding; it takes an hour.
4. Why is my bill so high?
Most likely an accelerator instance left running, or real-time serving provisioned for peak where batch would have worked. Check idle resources first, then storage accumulation, then egress. Training compute is rarely the culprit.
5. What are spot instances and should I use them?
Interruptible capacity at sixty to ninety percent less, which the provider can reclaim at short notice. Use them for any training that checkpoints and can resume, which should be all of it. Do not use them for serving traffic that users are waiting on.
6. Is serverless inference worth it?
For low or spiky traffic, yes — you pay per request and scale to zero. The cold start after a quiet period makes it a poor fit when a user is waiting, so it suits internal and background workloads better than interactive ones.
7. Is my data used to train the provider's models?
Generally no on enterprise tiers of the major providers, and frequently yes on consumer tiers. Confirm it for the specific service and tier, and make sure staff know which account to use for work data, because that is where real leaks come from.
8. What if data cannot leave my country?
Verify processing region rather than storage region, since managed services do not always process where they store. Where no provider satisfies the requirement, a self-hosted open model inside your own boundary is a legitimate architecture and a large part of why open models matter commercially.
9. How do I avoid lock-in?
Containers for your logic, open data formats, an interface between your code and any proprietary service, and code in your own version control. Skip multi-cloud unless something concrete requires it, and spend real switchability effort only on hosted model APIs, where you will actually use it.
10. Should I buy GPUs instead of renting?
For steady high-volume inference running continuously, owned or long-term reserved hardware is frequently cheaper, and the crossover arrives sooner than cloud pricing suggests. For variable and experimental work, renting wins clearly. Calculate it rather than assuming either way.
11. Is automated model search worth using?
As a baseline, yes. Point it at a tabular dataset and you learn in an hour what an easy solution scores, which tells you whether a hard one is justified. Treat beating it as your bar rather than your goal.
12. Which platform should I learn for my career?
Whichever your employer or client uses. The concepts transfer almost entirely, and only console layouts and service names differ, which is days of work once the concepts are in place. The premium catalogue covers SageMaker, Azure ML and Vertex AI hands-on.
Vocabulary you can now use precisely
| Term | Meaning in one line |
|---|---|
| Managed AI API | A pretrained model behind an endpoint, no data or training needed |
| ML platform | Managed notebooks, training, registry, endpoints, pipelines, monitoring |
| Hosted foundation model | Access to a large pretrained model, priced per token or image |
| Model gateway | One interface exposing several model families inside your cloud account |
| Managed training job | Provisions, runs, writes the model and shuts the machine down |
| Spot / preemptible | Interruptible capacity at a large discount |
| Serverless inference | Pay per request, scales to zero, cold start on first request |
| Egress | Charges for data leaving a provider's network |
| Data residency | Requirement that data be stored, and processed, in a given region |
| Private endpoint | A network path to a service that avoids the public internet |
| Model extraction | Approximating a model by querying its API repeatedly |
| Membership inference | Determining whether a record was in the training data |
| Data gravity | The practical difficulty of moving data, and the real source of lock-in |
And now?
You can read a cloud AI catalogue, pick the right category, predict where the bill will come from, and answer a security review accurately. That covers most of what the platforms will ask of you.
Validate it: take the 5-question quiz →
Continue free: Ethics of AI covers what your obligations are regardless of provider.
Go deeper: the premium catalogue covers AWS SageMaker, Azure Machine Learning and GCP Vertex AI hands-on, with a verifiable certificate after a 40-question examination.