SysPad › Lessons › AI and machine learning lessons › SageMaker Endpoint

How SageMaker Endpoint works

Managed real-time ML inference endpoint. Each instance runs parallel model copies; throughput is limited by inference latency × concurrency. Scale out with more instances or up with GPU/Inferentia types.

Training and serving your own machine-learning model usually means juggling GPU servers, environments, and deployment plumbing. SageMaker is the managed workshop for all of it: write the notebook, fire off a training job on GPUs that vanish when done, and put the result behind an endpoint, without running any of that infrastructure yourself.

What you will learn

What stages of the ML lifecycle does SageMaker cover?
How do managed training jobs save you from running GPU servers?
What are the ways to deploy a model, and how does each bill?
When do you use SageMaker versus Bedrock?

Build, train, deploy in one place

SageMaker is an end-to-end ML platform. You build in managed notebooks (SageMaker Studio), train with managed jobs on CPU or GPU fleets, and deploy the trained model for inference, all in one service. The key point: you bring and own the model.

Managed training jobs

A training job provisions the ml.* instances you ask for (often GPU types like ml.g or ml.p), runs your training code, writes the model artifact to S3, then tears the instances down. You pay for the run, not for idle GPUs sitting around between experiments.

⚓ Teams training custom models: recommendation engines, computer vision, fraud detection, forecasting, and fine-tuning open models on their own data.

Deploying: endpoints and the idle trap

There are several ways to serve a model: a real-time endpoint (warm instances, low latency), serverless inference (scales to zero, cold-start trade-off), asynchronous (queued, large payloads), and batch transform (offline scoring of a dataset).

SageMaker vs Bedrock

Both are AWS AI services with opposite starting points. SageMaker is for your own models, you train and host them, controlling instances and cost. Bedrock serves managed foundation models (Claude, Llama, and more) through an API with no infrastructure, billed per token.

Flashcards

What three lifecycle stages does SageMaker cover?: Build (notebooks), train (managed jobs), and deploy (inference), for models you own.
How does a managed training job bill?: It provisions ml.* instances, runs, then tears them down; you pay only for the training run.
What is the cost trap with a real-time endpoint?: It bills per instance-hour while up, even when idle; spiky/low traffic is often cheaper on serverless or batch.
Name the SageMaker deployment options.: Real-time endpoint, serverless inference, asynchronous, and batch transform.
SageMaker vs Bedrock in one line?: SageMaker = your own models, you host them; Bedrock = managed foundation models via API, per token.

Sources

Open the SysPad canvas · Official SageMaker Endpoint documentation