huggingface models on amazon sagemaker - pulumi iac

how-to.md

How To

1. We’ll make a directory

2. And create a new project from pulumi templates

mkdir ai && cd ai
pulumi new sagemaker-aws-python

a. Give the project a name
b. Name our stack
c. Choose your aws region
d. Now Pulumi will setup our python virtual environment

It's that easy let's deploy!

3. Run the pulumi up command to deploy

pulumi up

There it is! Our very own Large Language Model endpoint!

Now let's try a prompt with python:

1. be sure to activate your virtual environment

source venv/bin/activate.fish

2. Create a small test python script

import json, boto3, argparse

def main(endpoint_name):
    client = boto3.client('sagemaker-runtime', region_name='us-east-1')
    payload = json.dumps({"inputs": "In 3 words, name the biggest mountain on earth?"})
    response = client.invoke_endpoint(EndpointName=endpoint_name, ContentType="application/json", Body=payload)
    print("Response:", json.loads(response['Body'].read().decode()))

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("endpoint_name")
    main(parser.parse_args().endpoint_name)

3. Send the test python prompt!

python3 test.py (pulumi stack output EndpointName)

Like and subscribe for more Machine Learning content. Go make stuff, break stuff, and automate it all with Pulumi.