Skip to main content

Home

Using Azure AD for Databricks REST API authentication

You can use the Azure active directory for Databricks REST API authentication instead of the usual Personal Access Token authentication. Do the following:

  1. Create a service principal.

    1. From the Azure portal, log on to your Azure Account.

    2. Select Azure Active Directory > App Registrations > New Registrations and register your app. You must also add the registered app to the admins group as shown in Step 3.

    3. Go to Certificates & secrets and click on New client secret.

    4. Provide a description of the secret and a duration.

    5. Click Add. The client secret is displayed, which you must copy and keep handy.

  2. Add the service principal as a contributor (or Reader) to each workspace.

    1. Collect information on the following items before you assign the service principal to each workspace.

      Items

      Description

      personal-access-token

      Personal Access Token (PAT) that was used for managing workspace.

      databricks-instance

      URL of the corresponding workspace where you assign the service principal.

      application-id

      Application (client) ID of the application that was registered in the previous step.

      display-name

      Name of application that was created in the previous step.

    2. Use SCIM API to assign the created service principal to each workspace.

      curl --netrc -X POST -H "Authorization: Bearer <personal-access-token>" <databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals \
--header 'Content-type: application/scim+json' \
--data '{
"schemas": [ "urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal" ],
"applicationId": "<application-id>",
"displayName": "<display-name>"
}'

  3. Add the registered app to the admins group.

    1. Go to Workspace > Settings > Admin console > Groups > admins > Add users or Service principals.

    2. Add the registered app that was created in Step 1.

  4. Set properties of the app in Unravel:

    1. Stop Unravel. 

      <Unravel installation directory>/unravel/manager stop

    2. Set the following properties using the manager tool from the installation directory:

      <Unravel installation directory>/unravel/manager config databricks set-azure-ad --client <databricks-client-id> --tenant <databricks-tenant-id> --secret <databricks-client-secret>

      For example:

      <Unravel installation directory>/unravel/manager config databricks set-azure-ad --client 701c561b-f48a-4673-9fa9-e7c4c5550c42 --tenant bc745a0d-f282-4e99-b95f-1ecb477a209e --secret EQ6_YCTT506gWV1Z.DXHJU1HW~I24HZCX_

      Properties

      Description

      com.unraveldata.databricks.client_id

      Specify the client ID

      com.unraveldata.databricks.client_secret

      Specify the client secret

      com.unraveldata.databricks.tenant_id

      Specify the Tenant ID

    3. Apply the changes. 

      <Unravel installation directory>/unravel/manager config apply

    4. Start Unravel. 

      <Unravel installation directory>/unravel/manager start
In this section: