SageMaker Endpoints and Inference

Duration: 5 min

This module delves into Amazon SageMaker Endpoints and Inference, a critical component for deploying machine learning models into production. Understanding how to effectively deploy and manage endpoints is essential for ensuring that your models are scalable, reliable, and performant. This module covers the creation, management, and monitoring of SageMaker endpoints, as well as best practices for inference.

Creating a SageMaker Endpoint

To deploy a machine learning model on SageMaker, you first need to create an endpoint. An endpoint is a hosted service that allows you to make predictions using your trained model. The process involves creating a model, configuring an endpoint configuration, and finally deploying the endpoint.

import boto3
from sagemaker.model import Model

# Initialize boto3 session
session = boto3.Session()

sagemaker_client = session.client('sagemaker')

# Specify the model details
model_name = 'my-model'
image = 'your-ecr-image-uri'
model_data = 's3://your-bucket/model.tar.gz'
role = 'your-iam-role-arn'

# Create a SageMaker Model
sagemaker_model = Model(image=image, model_data=model_data, role=role, name=model_name)
sagemaker_model.deploy(initial_instance_count=1, instance_type='ml.m5.large')

Try it in Google Colab:

{'EndpointArn': 'arn:aws:sagemaker:us-west-2:123456789012:endpoint/my-model-endpoint'}

Invoking a SageMaker Endpoint for Inference

Once the endpoint is deployed, you can invoke it to make predictions. This involves sending a request to the endpoint with input data and receiving the model's predictions in return. SageMaker provides a runtime client to facilitate this process.

import boto3
import json

# Initialize boto3 runtime client
runtime = boto3.client('sagemaker-runtime')

# Specify the endpoint name
endpoint_name ='my-model-endpoint'

# Prepare the input data
input_data = json.dumps({"instances": [[1.0, 2.0, 5.0]]})

# Invoke the endpoint
response = runtime.invoke_endpoint(EndpointName=endpoint_name,
                                   ContentType='application/json',
                                   Body=input_data)

# Extract and print the prediction
result = json.loads(response['Body'].read())
print(result)

💡 Tip: Ensure that the content type and data format match the requirements of your model when invoking the endpoint. Mismatched formats can lead to errors or incorrect predictions.

Key Concepts

Check Your Understanding