Advent of 2023, Day 8 – Using Lakehouse REST API
This article is originally published at https://tomaztsql.wordpress.com
In this Microsoft Fabric series:
- Dec 01: What is Microsoft Fabric?
- Dec 02: Getting started with Microsoft Fabric
- Dec 03: What is lakehouse in Fabric?
- Dec 04: Delta lake and delta tables in Microsoft Fabric
- Dec 05: Getting data into lakehouse
- Dec 06: SQL Analytics endpoint
- Dec 07: SQL commands in SQL Analytics endpoint
Now that we explored the lakehouse through the interface and workspaces, let’s check today, how can we use REST API. Microsoft Fabric Rest API defines a unified endpoint for operations.
This endpoint is https://api.fabric.microsoft.com/v1/workspaces/{workspaceId}/items
Both WorkspaceID or LakehouseID should be replaced accordingly. For example, retrieving information on the lakehouse, you should be using GET method and calling “https://api.fabric.microsoft.com/v1/workspaces/1860beeexxxxxxx8cc-xxxxxxxxd699b92e1/lakehouses/a574d1a3-f10e-xxxxxxxxxxxxxxx7128f”.
There are many more REST methods availble, and these are (source: Microsoft Learn)
Some of the APIs are still in preview, like this listing all the tables using API – via Microsoft Learn.
But for the sake of brevity, let’s just call a GET method and retrieve some additional information on lakehouse. I have used Python code, as well as PostMan.
Note (!): authentication should be done through the Admin portal and Graph.
import sys
import json
import logging
import requests
import msal
config = json.load(open(sys.argv[1]))
# Create a preferably long-lived app instance which maintains a token cache.
app = msal.ConfidentialClientApplication(
config["client_id"], authority=config["authority"],
client_credential=config["secret"],
# token_cache=... # Default cache is in memory only.
# You can learn how to use SerializableTokenCache from
# https://msal-python.rtfd.io/en/latest/#msal.SerializableTokenCache
)
# The pattern to acquire a token looks like this.
result = None
# Firstly, looks up a token from cache
# Since we are looking for token for the current app, NOT for an end user,
# notice we give account parameter as None.
result = app.acquire_token_silent(config["scope"], account=None)
if not result:
logging.info("No suitable token exists in cache. Let's get a new one from AAD.")
result = app.acquire_token_for_client(scopes=config["scope"])
if "access_token" in result:
# Calling graph using the access token
graph_data = requests.get( # Use token to call downstream service
config["endpoint"],
headers={'Authorization': 'Bearer ' + result['access_token']}, ).json()
print("Graph API call result: ")
print(json.dumps(graph_data, indent=2))
else:
print(result.get("error"))
print(result.get("error_description"))
print(result.get("correlation_id")) # You may need this when reporting a bug
api_response = requests.get("https://api.fabric.microsoft.com/v1/workspaces/1860beeexxxxxxx8cc-xxxxxxxxd699b92e1/lakehouses/a574d1a3-f10e-xxxxxxxxxxxxxxx7128f")
data = api_response.text
parse_json = json.loads(data)
print(parse_json)
Or you can get the results in a program like Postman, Insomnia and other:
Tomorrow we will look into building custom environment for shared libraries, Spark compute settings, and resources for notebooks and Spark job definitions.
Complete set of code, documents, notebooks, and all of the materials will be available at the Github repository: https://github.com/tomaztk/Microsoft-Fabric
Happy Advent of 2023!
Thanks for visiting r-craft.org
This article is originally published at https://tomaztsql.wordpress.com
Please visit source website for post related comments.