Skip to main content

Home

Using Azure AD for Databricks REST API authentication

You can use the Azure active directory for Databricks REST API authentication instead of the usual Personal Access Token authentication. Do the following:

  1. Create a service principal.

    1. From the Azure portal, log on to your Azure Account.

    2. Select Azure Active Directory > App Registrations > New Registrations and register your app. You must also add the registered app to the admins group as shown in Step 3.

    3. Go to Certificates & secrets and click on New client secret.

    4. Describe the secret and its duration.

    5. Click Add. The client's secret is displayed, which you must copy and keep handy.

  2. Add the service principal as a contributor (or Reader) to each workspace.

    1. Collect information on the following items before you assign the service principal to each workspace.

      Items

      Description

      personal-access-token

      Personal Access Token (PAT) that was used for managing workspace.

      databricks-instance

      URL of the corresponding workspace where you assign the service principal.

      application-id

      Application (client) ID of the application that was registered in the previous step.

      display-name

      Name of application that was created in the previous step.

    2. Use SCIM API to assign the created service principal to each workspace.

      curl --netrc -X POST -H "Authorization: Bearer <personal-access-token>" <databricks-instance>/api/2.0/preview/scim/v2/ServicePrincipals \
--header 'Content-type: application/scim+json' \
--data '{
"schemas": [ "urn:ietf:params:scim:schemas:core:2.0:ServicePrincipal" ],
"applicationId": "<application-id>",
"displayName": "<display-name>"
}'

  3. Add the registered app to the admins group.

    1. Go to Workspace > Settings > Admin console > Groups > admins > Add users or Service principals.

    2. Add the registered app that was created in Step 1.

  4. Set properties of the app in Unravel.

    1. Stop Unravel. 

      <Unravel installation directory>/unravel/manager stop

    2. Run the following command and set the below properties using the manager tool.

      <Unravel installation directory>/unravel/manager config databricks set-azure-ad --client <databricks-client-id> --tenant <databricks-tenant-id>

      You are prompted to enter the client's secret. Type the secret, which will be masked, and press ENTER.

      Properties

      Description

      com.unraveldata.databricks.client_id

      Specify the client ID

      com.unraveldata.databricks.client_secret

      Specify the client secret

      com.unraveldata.databricks.tenant_id

      Specify the Tenant ID

    3. Apply the changes. 

      <Unravel installation directory>/unravel/manager config apply

    4. Start Unravel. 

      <Unravel installation directory>/unravel/manager start
In this section: