Using Interactive Precheck utility

You can use the Interactive Precheck utility to validate the required configurations before installing Unravel and reuse that information to bootstrap.

Download, install, and run the Interactive Precheck utility

Interactive precheck is available as:

Standalone Interactive package

To download and run the Interactive Precheck utility from a Standalone package, do the following:

Download the Interactive Precheck package.

curl -v https://preview.unraveldata.com/unravel/RPM/4.7.x/unravel-healthcheck-4.7.x.x.tar.gz -o unravel-healthcheck-4.7.x.x.tar.gz -u username:password

Extract the unravel-healthcheck tar.

tar -xvzf unravel-healthcheck-release-4.7.x.x.xxxx.tar.gz

Change directory to healthcheck.
```
cd <Installation-directory>/unravel/versions/4.7.x.x.xxxx/healthcheck
```
For example: cd /opt/unravel/versions/4.7.x.x.xxxx/healthcheck
Run the Interactive Precheck script.
```
./precheck.sh
```

Interactive Precheck as part of Unravel installation package

To download and run the Interactive Precheck utility, which comes as part of Unravel installation, do the following:

Follow the instructions to download and deploy the Unravel package. Refer to Unravel Installation topics.
Before you run the setup command, run the precheck.sh script from unravel/versions/X.Y.Z/healthcheck/

Use the Interactive Precheck utility

When you run the Interactive Precheck utility, it will prompt a series of questions to verify your environment and get the basic configuration information. This information is verified. The last section of interactive precheck allows you to generate a configuration file with your answers. You can then use this configuration file when calling setup to configure Unravel with those answers.

Answering questions	Default values and previous answers are shown in square brackets as follows. Press ENTER to leave the response as is: Some questions [Answer]: Yes/No answers identified by (y/n) are not case sensitive. y, yes, n, no are accepted and prompted again in case of an incorrect answer. You are prompted for questions of integers again if you provide a non-integer answer.
Reusing previous answers	All your answers are saved immediately. You can come back later to complete the process. When you run precheck again, it will prompt you to confirm if you want to reuse previous answers and jump to a specific section. For example: -- Reuse existing answers /tmp/unravel-interactive-precheck already exists, do you want to reuse existing answers? (y/n) [Yes]: y Do you want to jump back to a specific check (y/n) [No]: y Reuse existing answers: Yes: will keep the previous answers. No: will delete all answers and restart with the defaults. If you answered Yes to reuse answers, jump to a specific check: Yes: will show you all the sections and allow you to jump to a specific check. No: will start from the beginning.
Jumping to a specific section or exiting	At any point during this process, you can press CTRL-C to bring a menu that lets you move back and forth between the different sections or exit. For example: -- Interactive precheck navigation Jump to check 1- Beginning 2- General 3- External database 4- Extra packages 5- Kerberos 6- Certificate trust chain 7- Hadoop cluster manager 8- YARN Resource Manager 9- Unravel HTTPS 10- Unravel UI port 11- Unravel directories 12- Email configuration 13- Full precheck 14- Bootstrap 15- Exit (or CTRL-C) Select one of the above [Exit (or CTRL-C)]: Answering 1 through 14 will move to that section and continue until the end. Answering 15 or doing CTRL-C again will exit interactive precheck.
Answering failed checks	When a check fails, you are given the option to change the answers and test again or ignore the failure and move to the next section. Answering Yes will bring you back to the beginning of the sections. Answering No will move on to the next section. You can come back to it later after resolving the cause of failure.

Bootstrapping an Unravel installation with Interactive Precheck

The responses that you have provided for the configuration information are used to generate a configuration file. You can use this configuration file when you run the setup command to configure Unravel with those answers.

Unravel node type (setup --enable-edge, --enable-core)
Platform type (setup --enable-emr, --enable-databricks, etc.)
External database (setup --external-database)
Extra packages like JDBC drivers (setup --extra)
Custom location for Unravel data (setup --data-directory)
Trusted certificates (manager config tls trust)
TLS configuration for the unravel UI (manager config tls)
Kerberos configuration (manager config kerberos)
Automatic Hadoop configuration (manager config auto)
Email configuration (manager config email)

For example:

-- Unravel bootstrap configuration

   Generate a unravel bootstrap configuration file? (y/n) [Yes]: 

   File name and location: [/tmp/unravel-interactive-precheck/unravel-bootstrap.yaml]: 
   
   You can use the bootstrap configuration when running setup by using: --bootstrap /tmp/unravel-interactive-precheck/unravel-bootstrap.yaml

Interactive Precheck checks

The Interactive Precheck includes the following checks:

General

This section covers general information about your Unravel install. You are prompted for the following:

Data platform you want to monitor.
For Hadoop: type of Unravel node you want to configure.
For edge nodes: core node location and test connectivity.

-- General information
Which data platform are you installing for?
   1- Hadoop
   2- EMR
   3- HDI
   4- Databricks
   5- Dataproc

   Select one of the above [Hadoop]: 

Which type of Unravel node is this?
   1- Single node installation
   2- Core node
   3- Edge node

Select one of the above [Single node installation]: 
-- Core node - Log receiver information
   Hostname [None]: some.host.exampl
   Port (integer) [4043]: 
   TLS (y/n) [No]: y

   Attempting to connect to: https://some.host:4043/status

   Testing connection...
   Connection information:
     - Resolved IP:  89.163.146.49
     - Reverse lookup: ('app-002-de.some.host', [], ['89.163.146.49'])
     - Connection:     [Errno 111] Connection refused

   LR access: ERROR
   - ERROR: Fail to verify the connection

Database configuration

This check allows you to configure database-related information and an external database for Unravel.

-- Database configuration
 Configure an external database? (y/n) [No]:
 ##If you answer “No”, Unravel will use an Unravel managed database.
 ## If you answer “Yes”, you are further prompted for database information and test connectivity to that database.
 ##If you have answered “No” for the external database, you are further prompted to confirm if you want to use MySQL or MariaDB.

 Will you be using an Unravel managed MySQL or MariaDB? (y/n) [No]: 
 ##If you answer “No”,  the PostgreSQL bundled with Unravel is used.
 ##If you answer “Yes”, the “Extra packages” section searches for the required MySQL/MariaDB server tarball.

 Will Unravel connect to a MySQL or MariaDB database (ex: hive metastore) ? (y/n) [No]: 
 ##If you answer “Yes”, the “Extra packages” section searches for the required jdbc drivers.

Extra packages

The Extra packages check shows if you use Unravel managed MySQL/MariaDB or need JDBC drivers. Else, this check is automatically skipped.

-- Extra package location

*** JDBC drivers are required for Unravel managed MySQL or MariaDB.
*** Database software package is required for Unravel managed MySQL or MariaDB.

External package location [None]: /<my-extra-packages>
##This is the path to the directory where the required packages are located.

If the required packages are located, then the following message is shown:

The following packages will be installed:
   Database server: /my-extra-packages/mysql-5.7.27-linux-glibc2.12-x86_64.tar.gz
   JDBC driver:
     - /my-extra-packages/mysql-connector-java-5.1.48.tar.gz

 External package: Ok

If the required packages are not found, then the following error message is showing:

External package: ERROR
 - ERROR: Couldn't find jdbc drivers in /my-extra-packages
 - ERROR: Looked for:
   mysql-connector-java-*.tar.gz
   mysql-connector-java-*.jar
   mariadb-java-client-*.jar
 - ERROR: Couldn't find database server package in /my-extra-packages
 - ERROR: Looked for:
   mysql-*-linux-glibc2.12-x86_64.tar.gz
   mariadb-*-linux-x86_64.tar.gz

Kerberos

This check allows you to enable and configure Kerberos to access your Hadoop cluster.

-- Kerberos configuration
  Configure unravel to use kerberos? (y/n) [No]: 
  ##If you answer “No”, Kerberos will not be configured for Unravel.
  ##If you answer “Yes”, you are further prompted for keytab location and principal. The information is then validated for accuracy.

Trust chain certificates

This check allows you to add root and intermediate certificates to validate the trust chain to establish the connection using TLS. Currently, only pem files are supported.

-- TLS certificate trustchain
  Add trusted certificates? (y/n) [No]:  
  ##If you answer “No”, certificates will not be added.
  ##If you answer “Yes”, All the certificates found at the specified location are imported. Wildcards can be used.

Hadoop cluster manager

This check allows you to specify a cluster manager (Ambari or Cloudera Manager) to configure Unravel automatically.

-- Hadoop cluster manager (Ambari or Cloudera Manager only)
  
  Configure a cluster manager? (y/n) [No]: 
  ##If you answer “Yes”, you are prompted for the full URL and credentials to test the connection.

Yarn resource manager

This allows you to test connectivity to the YARN resource manager. The connection test uses the Kerberos information and trusted certificates if it is configured.

-- YARN Resource Manager Access

  Detect YARN configuration (y/n) [Yes]: 
  ##If you answer “Yes”, Yarn information is obtained from the local configuration files if they are available. You are further prompted for information if required.
  ##If you answer “No”, you are further prompted for yarn connection information.

Unravel HTTPS

This check allows you to configure and test HTTPS for the unravel UI. This check prompts you for certificate, key, password, and hostname details that are used to access Unravel.

-- TLS configuration

Note

You must provide the certificate and key in PEM format.

The information provided is verified for the following:

If the Key and Certificate match
If the certificate is valid
If the certificate applies for the provided hostname

Unravel UI port

This check allows you to set the Unravel UI port and verify the connectivity.

-- Unravel default port
  
  Port number (integer) [3000]: 

  Do you want to test if the port is accessible? (y/n) [Yes]: 
    
  This will open port 3000 and listen for connection for 120 seconds.
  Use your browser to test if the Unravel UI will be accessible on that port.
    
  We have detected the following hostnames:
  - some.host.example
    
 Browse to: http://some.host.example:3000
    
 ATTENTION: This address is an example. You should test with the URL that will be used to access Unravel.

A connection on port 3000 is tried and established. If the connection is successful, Unravel Port Test: OK is shown on the browser, and Unravel port: Reached is shown on the server.

Unravel directories

This check allows you to set a custom data directory and verify the access if the directories exist. You will always find the software location where you deploy the Unravel binaries. In this check, only the space and access are tested. That data location that you have configured will be used.

-- Unravel directories

  Software [/opt/unravel]: 

  Data [/opt/unravel/data]: 

  Directories: ERROR
  - OK: 33 GB of free disk space for software.
  - ERROR: SYSH0026: Space for data 33 GB is low, recommended minimum is 100 GB.

Email configuration

This check allows you to configure and test email. You are prompted for host and credentials, and the following items are tested:

Connectivity
Authentication, only if provided.
Optional: Send test mail.

-- Mail server (SMTP) configuration

Full precheck

This option allows you to run the full precheck using some of the provided information. Additional tests like user limits, CPU, and memory are run this way.

-- Full precheck

Run Interactive Precheck automatically with existing answers

This can be used to quickly verify previously answered questions (For example, test X failed, was fixed, and re-run everything to ensure it works). Moreover, in the context of automation, you can generate an answer file and run an interactive precheck with it to verify everything and get a bootstrap configuration.

To obtain the default answers to all the questions:
```
precheck.sh --show-defaults
```
To run without user interaction using existing answers:
```
precheck.sh --headless
```

In this section:

Home