Advent of 2024, Day 20 – Microsoft Azure AI – Models and endpoints in Azure AI Foundry

This article is originally published at https://tomaztsql.wordpress.com

In this Microsoft Azure AI series:

1. Dec 01: Microsoft Azure AI – What is Foundry?
2. Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
3. Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
4. Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
5. Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
6. Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
7. Dec 07: Microsoft Azure AI – Speech service in AI Services
8. Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
9. Dec 09: Microsoft Azure AI – Speech SDK with Python
10. Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
11. Dec 11: Microsoft Azure AI – Language and Translation Python SDK
12. Dec 12: Microsoft Azure AI – Vision and Document AI Service
13. Dec 13: Microsoft Azure AI – Vision and Document Python SDK
14. Dec 14: Microsoft Azure AI – Content safety AI service
15. Dec 15: Microsoft Azure AI – Content safety Python SDK
16. Dec 16: Microsoft Azure AI – Fine-tuning a model
17. Dec 17: Microsoft Azure AI – Azure OpenAI service
18. Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
19. Dec 19: Microsoft Azure AI – Azure AI Foundry management center

Models from the model catalog can be deployed using programming languages or using the Foundry studio.

Model deployment has two types: Deploy from the base model or deploy from the fine-tuned model. The difference is that fine-tuned model is model taken from the model catalog and later tuned to an additional dataset, as the base model is the model as it is available in Azure AI Foundry.

Each model has all the necessary detail information, as well all the metrics to check number of requests, prompt token counts, number of requests and additional information on model usage.

Model from the model catalog can also be deployed as a serverless API with pay-as-you-go billing. This kind of deployment provides a way to consume models as an API without hosting them on your subscription. Models deployed in Azure Machine Learning (AMLS) and Azure AI Foundry in Serverless API endpoints support the Azure AI Model inference API (read more here: https://learn.microsoft.com/en-us/azure/ai-studio/reference/reference-model-inference-api ) and can be used by developers from inference or consuming the predictions.

Deployment with Python SDK is straightforward with Python and serverless API endpoint can be set with the MLClient:

from azure.ai.ml import MLClient
from azure.identity import InteractiveBrowserCredential
from azure.ai.ml.entities import MarketplaceSubscription, ServerlessEndpoint

client = MLClient(
    credential=InteractiveBrowserCredential(tenant_id="<tenant-id>"),
    subscription_id="<subscription-id>",
    resource_group_name="<resource-group>",
    workspace_name="<project-name>",
)

And an endpoint for the serverless:

endpoint_name="My_endpoint"

serverless_endpoint = ServerlessEndpoint(
    name=endpoint_name,
    model_id=model_id
)

created_endpoint = client.serverless_endpoints.begin_create_or_update(
    serverless_endpoint
).result()

endpoint_keys = client.serverless_endpoints.get_keys(endpoint_name)
print(endpoint_keys.primary_key)
print(endpoint_keys.secondary_key)

Tomorrow we will look into the prompt flow in Azure AI Foundry.

All of the code samples will be available on my Github.

Thanks for visiting r-craft.org
This article is originally published at https://tomaztsql.wordpress.com
Please visit source website for post related comments.