Skip to main content

Home

Configuring Unravel on Github

Unravel offers Continuous Integration and Continuous Deployment (CI/CD) integration for the GitHub platform. This new capability enhances data application development by integrating Unravel's intelligence and monitoring tools into the CI/CD pipeline. This integration helps detect inefficiencies that reduce query performance and provides insights, such as reviewing using union instead of union all, using order by without limits, and usage of functions in predicate insights before queries reach production. By using this integration, you can save costs and enhance query performance.

What to expect

  • Unravel CI/CD integration allows for early detection and prevention of performance issues.

  • Developers can proactively address bottlenecks, anomalies, and failures before they impact users.

  • Teams can meet financial and performance goals by analyzing critical metrics during development.

  • Bottlenecks are identified and resolved promptly, reducing cycle times.

  • Real-time visibility helps in making data-driven decisions.

Configuring Unravel on Github

There are three steps to configure Unravel on Github:

  1. Configure Secrets

  2. Configure Variables

  3. Add the required files

Configuring the Secrets

To configure CI/CD integration into your GitHub repository:

  1. In your GitHub repository, go to Settings.

    Settings.png

  2. From the left pane, click Secrets and Variables > Actions. The Actions secrets and variables window is displayed.

    Secrets-actions.png

    Configure the following three secrets under secrets:

User GitHub Token

  1. Click the user profile icon in the top right corner and select Settings.

    profile-settings.png

  2. Click Developer settings from the left pane. The GitHub Apps window opens.

    Developer-settings.png

  3. Click the Personal access tokens drop-down and select Generate new token.

    img-6-github.png

  4. Select the expiration period.

  5. Select the required access. The following access permissions are mandatory:

    • workflow

    • user

    • write

    PAT-classic-window.png

  6. Click Generate token. The token is generated. Copy the token.

Unravel JWT Token

  1. From the Unravel UI, click the Configuration icon > API tokens. The API tokens window opens.

    API-token.png

  2. Click New. The New API Token dialog box appears.

  3. Enter the following details:

    • Client ID: unique ID for the API token reference.

    • Role: The role will be admin.

    New-api-token-window.png

  4. Select the Admin role and click Create.

  5. Copy the token.

    img-github-3.png
Unravel URL

Copy the IP address of the machine where Unravel is hosted.

For example: 

<unravel_host_url>/api/v1/ds/api/v1/sql/analyze
Add the Secrets

  1. Under Secrets and Variables, click Actions > New Repository Secret. The New secret window opens.

  2. Provide a name for the secret.

  3. Paste the copied user GitHub token and click Add secret. The user token is added.

    New-secret-window.png

  4. Repeat steps 1-3 for the Unravel JWT token and Unravel URL, adding each as a new secret.

    Note

    Add the JWT token in this format: JWT <Paste the Copied Token>

Configuring the Variables

  1. In your GitHub repository, go to Settings.

  2. From the left pane, click Secrets and Variables > Actions. The Actions secrets and variables window opens.

  3. Click the Variables tab.

  4. Add the Variables by following these steps:

    1. Under Repository variables, click New Repository variable. The New variable window opens.

      New-variable.png

    2. Provide a name for the variable. For example, CICD_Platform.

    3. Under Value, paste your BigQuery account ID (the identifier of your BigQuery account) and click Add variable. The variable is added.

      Account.png
Add the Files

The Workflow file, Python Setup, and Python file should be added to the Github repository. Follow the steps given here to add the files.

Add the Workflow File

  1. Click the plus icon beside Go to file in the repository and create a new file.

  2. The workflow file should be created inside .github/workflows. Name the file as unravel-cicd-workflow.yaml. This naming convention should be followed.

    Note

    The unravel-cicd-workflow.yaml file is available here.

  3. Copy the YAML file provided by Unravel here and commit the changes directly to the main branch.

Add Python Setup File

  1. Click the plus icon beside Go to file in the repository and create a new file.

  2. This file should be created inside unravel-cicd-script folder. Name the file unravel-cicd-setup.py. This naming convention should be followed.

  3. Copy the unravel-cicd-setup.py file provided by Unravel here and commit the changes directly to the main branch.

    Note

    The unravel-cicd-setup.py file is available here.

Add Python File

  1. Click the plus icon beside Go to file in the repository and create a new file.

  2. This file should be created inside unravel-cicd-script/. Name the file unravel-cicd-executor.py. This naming convention should be followed.

    Note

    The unravel-cicd-executor.py file is available here.

  3. Copy the unravel-cicd-executor.py file provided by Unravel. Propose the changes and create a pull request.

    As the pull request is created, a GitHub workflow is triggered. This configures the CI/CD in your GitHub repository.

    The workflow initializes the containers, installs jq, checks out the code, installs dependencies, and executes the Python script.

    checks.png

    After completion and all checks have passed, merge the pull request.

    You can delete this branch after the merge is completed.

Once merged and the scripts have been added to the GitHub repository, your CI/CD is configured, and you can continue executing SQL queries.

In this section: