Advent of 2024, Day 23 – Microsoft Azure AI – Tracing in Azure AI Foundry
This article is originally published at https://tomaztsql.wordpress.com
In this Microsoft Azure AI series:
- Dec 01: Microsoft Azure AI – What is Foundry?
- Dec 02: Microsoft Azure AI – Working with Azure AI Foundry
- Dec 03: Microsoft Azure AI – Creating project in Azure AI Foundry
- Dec 04: Microsoft Azure AI – Deployment in Azure AI Foundry
- Dec 05: Microsoft Azure AI – Deployment parameters in Azure AI Foundry
- Dec 06: Microsoft Azure AI – AI Services in Azure AI Foundry
- Dec 07: Microsoft Azure AI – Speech service in AI Services
- Dec 08: Microsoft Azure AI – Speech Studio in Azure with AI Services
- Dec 09: Microsoft Azure AI – Speech SDK with Python
- Dec 10: Microsoft Azure AI – Language and Translation in Azure AI Foundry
- Dec 11: Microsoft Azure AI – Language and Translation Python SDK
- Dec 12: Microsoft Azure AI – Vision and Document AI Service
- Dec 13: Microsoft Azure AI – Vision and Document Python SDK
- Dec 14: Microsoft Azure AI – Content safety AI service
- Dec 15: Microsoft Azure AI – Content safety Python SDK
- Dec 16: Microsoft Azure AI – Fine-tuning a model
- Dec 17: Microsoft Azure AI – Azure OpenAI service
- Dec 18: Microsoft Azure AI – Azure AI Hub and Azure AI Project
- Dec 19: Microsoft Azure AI – Azure AI Foundry management center
- Dec 20: Microsoft Azure AI – Models and endpoints in Azure AI Foundry
- Dec 21: Microsoft Azure AI – Prompt flow in Azure AI Foundry
- Dec 22: Microsoft Azure AI – Prompt flow using VS Code and Python
Tracing is a powerful tool that offers developers an in-depth understanding of the execution process of their generative AI applications. Though still in preview (in the time of writing this post), It provides a detailed view of the execution flow of the application and the essential information for debugging or optimisations.
Tracing with the Azure AI Inference SDK offers enhanced visibility and simplified troubleshooting for LLM-based applications, effectively supporting development, iteration, and production monitoring. Tracing follows the OpenTelemetry semantic conventions, capturing and visualizing the internal execution details of any AI application, enhancing the overall development experience.
Key advances of using tracing are:
- users can collect feedback logs and evaluate data to enrich trace data with additional insights
- get clear insights into the GenAI application
- gives options on sensitive data logging
- use Azure App insights
- simplified process for enabling traces for better improvements and productivity
When starting the tracing, create a new app or connect to an existing one.
The Kusto language for analysing the logs for attributes, content, spans, evaluation events and others. Excerpt for retrieving span attributes:
// Retrieve Gen AI span attributes within the specified date range
let gen_ai_spans = dependencies
| where isnotnull(customDimensions["gen_ai.system"])
| where timestamp between (datetime(2024-12-16T18:55:29.840Z) .. datetime(2024-12-23T18:56:18.214Z))
| project
operation_ParentId,
operation_Id,
customDimensions,
id,
duration,
success,
gen_ai_operation_start = timestamp,
name,
span_type = "GenAI",
gen_ai_operation_name = tostring(customDimensions["gen_ai.operation.name"]),
gen_ai_response_id = tostring(customDimensions["gen_ai.response.id"]),
gen_ai_response_model = tostring(customDimensions["gen_ai.response.model"]),
gen_ai_usage_input_tokens = tostring(customDimensions["gen_ai.usage.input_tokens"]),
gen_ai_usage_output_tokens = tostring(customDimensions["gen_ai.usage.output_tokens"]);
// Helper function to determine Gen AI role based on event name
let get_role = (event_name: string) {
iff(event_name == "gen_ai.choice", "assistant", split(event_name, ".")[1])
};
With Python SDK you can also collect and analyse the log traces.
import os
from opentelemetry import trace
# Install opentelemetry with command "pip install opentelemetry-sdk".
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage, CompletionsFinishReason
from azure.core.credentials import AzureKeyCredential
# [START trace_setting]
from azure.core.settings import settings
settings.tracing_implementation = "opentelemetry"
# [END trace_setting]
# Setup tracing to console
# Requires opentelemetry-sdk
span_exporter = ConsoleSpanExporter()
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(SimpleSpanProcessor(span_exporter))
trace.set_tracer_provider(tracer_provider)
# [START trace_function]
from opentelemetry.trace import get_tracer
tracer = get_tracer(__name__)
This code part can be added to your regular Python code in order for tracing to be activated.
Tomorrow we will look into Evaluation in Azure AI Foundry.
All of the code samples will be available on my Github.
Thanks for visiting r-craft.org
This article is originally published at https://tomaztsql.wordpress.com
Please visit source website for post related comments.
