Skip to main content

Home

Integrate database (On-prem)

Refer to the following instructions to integrate Unravel with a database other than the default PostgreSQL that is shipped with the installer.

You can integrate Unravel managed database as well as an external database for usage with Unravel. Unravel supports MySQL, MariaDB, and PostgreSQL (Postgres). Check the compatibility matrix for the versions of MySQL, MariaDB, and Postgres compatible with the corresponding Unravel platform version.

MySQL

You can integrate Unravel-managed MySQL as well as external MySQL with Unravel.

Unravel managed MySQL

Important

The following packages must be installed for fulfilling the OS level requirements for MySQL:

  • numactl-libs (for libnuma.so)

  • libaio (for libaio.so)

To integrate with Unravel managed MySQL, do the following:

  1. Create a mysql directory in /tmp. Provide permissions and make them accessible to the user who installs Unravel.

  2. Download the following tar files for MySQL server to /tmp/mysql directory:

    For MySQL 5.7.x

    • mysql-5.7.x-linux-glibc2.12-x86_64.tar.gz or

    • mysql-connector-java-5.1.x.tar.gz

    For MySQL 8.0.x

    • mysql-8.0.x-linux-glibc2.12-x86_64.tar.gz

    • mysql-connector-java-8.0.x.tar.gz

  3. Run the setup command, as an Unravel user, to set up Unravel with MySQL database:

    • For single cluster setup, you can run the command on the Unravel node as follows:

      <unravel_installation_directory>/unravel/versions/<Unravel version>/setup --extra /tmp/mysql

    • For multi-cluster setup, you must run the command on the core node as follows:

      <unravel_installation_directory>/versions/<Unravel version>/setup --enable-core --extra /tmp/mysql

This will configure and manage MySQL for usage with Unravel.

External MySQL

For configuring an external MySQLdatabase with Unravel, you must provide the JDBC connector. This can either be as a tar file or as a jar file.

To integrate Unravel with an external database, do the following:

  1. Create unravel schema and user on the target database where the unravel user should have full access to the schema.

    ## For example:
CREATE DATABASE unravel_mysql_prod;
CREATE USER 'unravel'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON unravel_mysql_prod.* TO 'unravel'@'localhost';

  2. Create a directory for MySQL in /tmp. Provide permissions and make them accessible to the user who installs Unravel.

  3. Add the JDBC connector to /tmp/<MySQL-directory>/<jdbcconnector> directory. This can be either a tar file or a jar file.

  4. Run the setup command, as an Unravel user, to set up Unravel with MySQL database:

    • For a single cluster setup, you can run the command on the Unravel node as follows:

      <unravel_installation_directory>/unravel/versions/<Unravel version>/setup --extra /tmp/<MySQL-directory> --external-database mysql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

    • For multi-cluster setup, you must run the command on the core node as follows:

      <unravel_installation_directory>/versions/<Unravel version>/setup --enable-core --extra /tmp/<mysql-directory> --external-database mysql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

This will configure and manage external MySQL for usage with Unravel.

Note

In the case of MySQL 8.0, the client does not support the authentication protocol requested by the server, and there can be a failed connection to the database. Run the following to resolve this authentication issue:

ALTER USER 'unravel'@'unravel-host' IDENTIFIED WITH mysql_native_password BY 'password';flush privileges;

Where unravel is your user, unravel-host is the URL, and password is your password.

MariaDB

You can integrate Unravel with MariaDB by providing a Java JDBC connector.

Unravel managed MariaDB

Important

The following packages must be installed for fulfilling the OS level requirements for MariaDB:

  • numactl-libs (for libnuma.so)

  • libaio (for libaio.so)

To configure with Unravel managed MariaDB, do the following:

  1. Create a mariadb directory in /tmp. Provide permissions and make them accessible to the user who installs Unravel.

  2. Download the following tar files for mariadb server to /tmp/mariadb directory:

    • mariadb-10.4.13-linux-x86_64.tar.gz

    • mariadb-java-client-2.6.0.jar

  3. Run the setup command, as an Unravel user, to set up Unravel with MariaDB database:

    • For a single cluster setup, you can run the command on the Unravel node as follows:

      <unravel_installation_directory>/unravel/versions/<Unravel version>/setup --extra /tmp/mariadb

    • For multi-cluster setup, you must run the command on the core node as follows:

      <unravel_installation_directory>/versions/<Unravel version>/setup --enable-core --extra /tmp/mariadb

This will configure and manage MariaDB for usage with Unravel.

External MariaDB

For configuring an external MariaDB database with Unravel, you must provide the JDBC connector. This can either be as a tar file or as a jar file.

To integrate Unravel with external MariaDB, do the following:

  1. Create unravel schema and user on the target database where the unravel user should have full access to the schema.

    ## For example:
CREATE DATABASE unravel_mariadb_prod;
CREATE USER 'unravel'@'localhost' IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON unravel_mariadb_prod.* TO 'unravel'@'localhost';

  2. Create a directory for MariaDB in /tmp. Provide permissions and make them accessible to the user who installs Unravel.

  3. Add the JDBC connector to /tmp/<MariaDB-directory>/<jdbcconnector> directory. This can be either a tar file or a jar file.

  4. Run the setup command, as Unravel user, to set up Unravel with an external MariaDB database:

    • For a single cluster setup, you can run the command on the Unravel node as follows:

      <unravel_installation_directory>/unravel/versions/<Unravel version>/setup --extra /tmp/<MariaDB-directory> --external-database mariadb <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>
##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

    • For multi-cluster setup, you must run the command on the core node as follows:

      <unravel_installation_directory>/versions/<Unravel version>/setup --enable-core --extra /tmp/<mariadb-directory> --external-database mariadb <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

This will set up Unravel with an external MariaDB.

PostgreSQL

You can configure external PostgreSQL, as well as, the Unravel managed PostgreSQL.

Follow the instructions in a special case, if you are using Unravel managed PostgreSQL and the Hive metastore is using MySQL.

Unravel managed PostgreSQL

To configure the Unravel managed PostgreSQL database, run the setup command:

  • For a single cluster setup, you can run the command on the Unravel node as follows:

    <unravel_installation_directory>/unravel/versions/<Unravel version>/setup

  • For multi-cluster setup, you must run the command on the core node as follows:

    <Unravel installation directory>/unravel/versions/<Unravel version>/setup --enable-core

This will set up Unravel with PostgreSQL.

Set up Unravel Managed PostgreSQL for Hive metastore with MySQL

If you are using Unravel managed PostgreSQL database and the Hive metastore is using MySQL, you can use one of the following options for setup.

Option 1

You can use this option if the Hive metastore is using MySQL version 5.7 or 8.0.

  1. Create a directory. For example, mysql directory in /tmp. Provide permissions and make them accessible to the user who installs Unravel.

  2. Download the following tar file (MySQL 5.7 or 8.0) to this directory and provide the directory path when you run setup to install Unravel with Unravel managed PostgreSQL.

    • For MySQL 5.7: mysql-connector-java-5.1.x.tar.gz

    • For MySQL 8.0: mysql-connector-java-8.0.x.tar.gz

  3. Run the setup command as follows:

    • Core node with Hadoop configuration

      <Unravel installation directory>/unravel/versions/<Unravel version>/setup --extra /path/to/directory with MySQL connector file
      ##Example:
/opt/unravel/versions/abcd.1234/setup --extra /tmp/mysql

    • Core node without Hadoop configuration

      <Unravel installation directory>/unravel/versions/<Unravel version>/setup --enable-core --extra /path/to/directory with MySQL connector file
      ##Example:
/opt/unravel/versions/abcd.1234/setup --extra /tmp/mysql --enable-core 
Option 2

You can use this option if the Hive metastore is using any other version other than MySQL version 5.7 or 8.0.

  1. Install Unravel.

  2. Copy the MySQL connector file to <unravel_installation_directory>/share/java/path.

  3. Provide permissions and make the location accessible to the user who installs Unravel. 

    chown -R username:groupname /path/to/directory with MySQL connector file
External PostgreSQL

To configure external PostgreSQL, do the following:

  1. Create unravel schema and user on the target database where the unravel user should have full access to the schema.

  2. Run the setup command from the installation directory:

    • For a single cluster setup, you can run the command on the Unravel node as follows:

      <unravel_installation_directory>/unravel/versions/<Unravel version>/setup --external-database postgresql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

    • For multi-cluster setup, you must run the command on the core node as follows:

      <unravel_installation_directory>/versions/<Unravel version>/setup --enable-core --external-database postgresql <HOST> <PORT> <SCHEMA> <USERNAME> <PASSWORD>

##The HOST, PORT, SCHEMA, USERNAME, PASSWORD are optional fields and are prompted if missing.

This will set up Unravel with an external PostgreSQL.

In this section: