Microsoft Azure Databricks
Before installing Unravel in Azure Databricks, check and ensure that the Unravel installation requirements are completed and follow the below instructions to install and configure Unravel:
1. Create Unravel VM and Azure Databricks resource
4. Install Unravel either with Interactive Precheck or manually
5. Configure Unravel Log Receiver
6. Connect Databricks cluster to Unravel
1. Create Unravel VM and Azure Databricks resource
Sign in to the Azure portal.
Select Virtual Machines > Add and enter the following information in the Basics tab:
Project Details
Subscription
Choose the applicable subscription.
Resource group
Create a new resource group or choose an existing one.
Instance Details
Virtual Machine Name:
The Unravel server name.
Region:
Select the Azure region.
Availability Options
Select
No infrastructure redundancy required
.Image
Select the appropriate image. Both
Centos-based 7.x
+ andRed Hat Enterprise Linux 7.x
+ are supported.Size
Click Change Size. In the modal, select Memory-optimized image with at least
128 GB memory
andPremium Disk support
, for example, E16s_v3 in East US 2)Administrator account
Authentication type
Select
password
orSSH Key
.Username and Password
Enter your VM login information.
Inbound Port Rules
Public inbound ports
Select Allow selected ports.
Selected Inbound ports
Select both
HTTPS
andSSH
.Click Next: Disks > and enter the following information in the Disks tab.
Disk Options
OS disk type: Select
Premium SSD
.
Data Disk
Click Create and attach a new disk.
Caution
This disk is formatted, so do not choose the Attach an existing disk option.
Enter a Name.
Select Source type
None (empty disk)
.Set Size to at least 512 GiB.
Account type: Select
premium SSD
.
Click Next: Networking > and enter the following information:
Virtual network: Create a new one or choose an existing one.
Subnet: Create a new or choose an existing one.
Public IP: Create a new one or choose an existing one.
Select Inbound ports: Select
HTTPS
andSSH
.
Click Review + create. Your deployment is now created.
Select Go to Resource > Networking > Inbound port rules > Add inbound port rule and include the following ports.
Name
Destination
Destination IP Address
Destination Port Ranges
Unravel_3000
IP Addresses
NIC Private IP
3000
Unravel_443
IP Addresses
NIC Private IP
443
Unravel_4043
IP Addresses
NIC Private IP
4043
Unravel_4443
IP Addresses
NIC Private IP
4443
Click OK.
Select Create a resource > Azure Databricks > Create. Go directly to step#3 if you already have workspaces.
Select Workspace name, Subscription, Resource group, Location, and Pricing tier.
Review VNET Peering options to connect Databricks with Unravel VM.
2. Download Unravel
3. Deploy Unravel
Unravel binaries are available as a TAR file or RPM package. You can deploy the Unravel binaries in any directory on the server. However, the user who installs Unravel must have the write permissions to the directory where the Unravel binaries are deployed.
After you extract the contents of the TAR file or RPM package, unravel
directory is created within the installation directory (<unravel_installation_directory
>), and Unravel will be available in <Unravel_installation_directory>/unravel. The directory layout will be unravel/versions/<Directories and files>
.
The following steps to deploy Unravel from a TAR file should be performed by a user, who will run Unravel.
Create an Installation directory.
mkdir
</path/to/installation/directory>
For example: mkdir /opt/
Extract Unravel tar file to the installation directory, which you have created in the first step. After you extract the contents of the TAR file,
unravel
directory is created within the installation directory.tar zxf unravel-
<version>
tar.gz -C</path/to/installation/directory>
For example: tar zxf unravel-4.7.x.x.tar.gz -C /opt
The unravel directory will be available within
/opt
Grant ownership of the directory to a user who will run Unravel.
chown -R username:groupname
</path/to/installation/directory>
For example: chown -R unravel:unravelgroup /opt/unravel
Important
The following steps to deploy Unravel from an RPM package should be performed by a root user. After the RPM package is deployed, the remaining installation procedures should be performed by the Unravel user.
Create an installation directory.
mkdir
</path/to/installation/directory>
For example: mkdir /usr/local/unravel
Run the following command:
rpm -i unravel-
<version>
.rpmFor example: rpm -i unravel-4.7.x.x.rpm
The unravel directory will be available in
/usr/local
If you want to provide a different location, use the --prefix command.
For example:
mkdir /opt/unravel rpm -i unravel-4.7.x.x.rpm --prefix /opt
The unravel directory will be available in
/opt
.Grant ownership of the directory to a user who will run Unravel. This user executes all the processes involved in Unravel installation.
chown -R
username
:groupname
</path/to/installation/directory>
For example:chown -R unravel:unravelgroup /usr/local/unravel
Continue with the installation procedures as unravel user.
4. Install Unravel
You can install Unravel either with Interactive Precheck or manually without Interactive Precheck.
Note
Unravel recommends installation with Interactive Precheck.
To install Unravel with Interactive precheck, you must run the Interactive Precheck utility to generate a bootstrap configuration file for installation, configure the Unravel log receiver, and then connect the Databricks cluster to Unravel.
5. Configure Unravel Log Receiver
6. Connect Databricks cluster to Unravel
Run the following steps to connect the Databricks cluster to Unravel.