Using the Interactive Precheck utility
You can use the Interactive Precheck utility to validate the required configurations before installing Unravel and reuse that information to bootstrap.
Download, install, and run the Interactive Precheck utility
Interactive precheck is available as:
Standalone Interactive package
To download and run the Interactive Precheck utility from a Standalone package, do the following:
Download the Interactive Precheck package.
curl -v https://preview.unraveldata.com/unravel/RPM/4.7.x/unravel-healthcheck-4.7.x.x.tar.gz -o unravel-healthcheck-4.7.x.x.tar.gz -u
username
:password
Extract the
unravel-healthcheck
tar.tar -xvzf unravel-healthcheck-release-4.7.x.x.xxxx.tar.gz
Change directory to
healthcheck
.cd <Installation-directory>/unravel/versions/4.7.x.x.xxxx/healthcheck
For example: cd /opt/unravel/versions/4.7.x.x.xxxx/healthcheck
Run the Interactive Precheck script.
./precheck.sh
Interactive Precheck as part of Unravel installation package
To download and run the Interactive Precheck utility, which comes as part of Unravel installation, do the following:
Follow the instructions to download and deploy the Unravel package. Refer to Unravel Installation topics.
Before you run the setup command, run the
precheck.sh
script fromunravel/versions/X.Y.Z/healthcheck/
Use the Interactive Precheck utility
When you run the Interactive Precheck utility, it will prompt a series of questions to verify your environment and get the basic configuration information. This information is verified. The last section of interactive precheck allows you to generate a configuration file with your answers. You can then use this configuration file when calling setup to configure Unravel with those answers.
Answering questions |
|
Reusing previous answers | All your answers are saved immediately. You can come back later to complete the process. When you run precheck again, it will prompt you to confirm if you want to reuse previous answers and jump to a specific section. For example: -- Reuse existing answers /tmp/unravel-interactive-precheck already exists, do you want to reuse existing answers? (y/n) [Yes]: y Do you want to jump back to a specific check (y/n) [No]: y Reuse existing answers:
If you answered Yes to reuse answers, jump to a specific check:
|
Jumping to a specific section or exiting | At any point during this process, you can press CTRL-C to bring a menu that lets you move back and forth between the different sections or exit. For example: -- Interactive precheck navigation Jump to check 1- Beginning 2- General 3- Database configuration 4- Extra packages 5- License 6- Certificate trust chain 7- Hadoop cluster manager 8- Unravel HTTPS 9- Unravel UI port 10- Unravel directories 11- Email configuration 12- Full precheck 13- Bootstrap 14- Exit (or CTRL-C) Select one of the above [Exit (or CTRL-C)]:
|
Answering failed checks | When a check fails, you are given the option to change the answers and test again or ignore the failure and move to the next section.
|
Bootstrapping an Unravel installation with Interactive Precheck
The responses you provided for the configuration information are used to generate a configuration file. You can use this configuration file when you run the setup command to configure Unravel with those answers.
Unravel node type (setup --enable-edge, --enable-core)
Platform type (setup --enable-emr, --enable-databricks, etc.)
External database (setup --external-database)
Extra packages like JDBC drivers (setup --extra)
Custom location for Unravel data (setup --data-directory)
Trusted certificates (manager config tls trust)
TLS configuration for the unravel UI (manager config tls)
Kerberos configuration (manager config kerberos)
Automatic Hadoop configuration (manager config auto)
Email configuration (manager config email)
For example:
-- Unravel bootstrap configuration Generate a unravel bootstrap configuration file? (y/n) [Yes]: File name and location: [/tmp/unravel-interactive-precheck/unravel-bootstrap.yaml]: You can use the bootstrap configuration when running setup by using: --bootstrap /tmp/unravel-interactive-precheck/unravel-bootstrap.yaml
Interactive Precheck checks
The Interactive Precheck includes the following checks:
General
This section covers general information about your Unravel install. You are prompted for the following:
Data platform you want to monitor.
For Hadoop: type of Unravel node you want to configure.
For edge nodes: core node location and test connectivity.
-- General information Which data platform are you installing for? 1- Hadoop 2- EMR 3- HDI 4- Databricks 5- Dataproc Select one of the above [Hadoop]: Which type of Unravel node is this? 1- Single node installation 2- Core node 3- Edge node Select one of the above [Single node installation]: -- Core node - Log receiver information Hostname [None]: some.host.exampl Port (integer) [4043]: TLS (y/n) [No]: y Attempting to connect to: https://some.host:4043/status Testing connection... Connection information: - Resolved IP: 89.163.146.49 - Reverse lookup: ('app-002-de.some.host', [], ['89.163.146.49']) - Connection: [Errno 111] Connection refused LR access: ERROR - ERROR: Fail to verify the connection
Database configuration
This check allows you to configure database-related information and an external database for Unravel.
-- Database configuration Configure an external database? (y/n) [No]: ##If you answer “No”, Unravel will use an Unravel managed database. ## If you answer “Yes”, you are further prompted for database information and test connectivity to that database. ##If you have answered “No” for the external database, you are further prompted to confirm if you want to use MySQL or MariaDB. Will you be using an Unravel managed MySQL or MariaDB? (y/n) [No]: ##If you answer “No”, the PostgreSQL bundled with Unravel is used. ##If you answer “Yes”, the “Extra packages” section searches for the required MySQL/MariaDB server tarball. Will Unravel connect to a MySQL or MariaDB database (ex: hive metastore) ? (y/n) [No]: ##If you answer “Yes”, the “Extra packages” section searches for the required jdbc drivers.
Extra packages
The Extra packages check shows if you use Unravel managed MySQL/MariaDB or need JDBC drivers. Else, this check is automatically skipped.
-- Extra package location *** JDBC drivers are required for Unravel managed MySQL or MariaDB. *** Database software package is required for Unravel managed MySQL or MariaDB. External package location [None]: /<my-extra-packages> ##This is the path to the directory where the required packages are located.
If the required packages are located, then the following message is shown:
The following packages will be installed: Database server: /my-extra-packages/mysql-5.7.27-linux-glibc2.12-x86_64.tar.gz JDBC driver: - /my-extra-packages/mysql-connector-java-5.1.48.tar.gz External package: Ok
If the required packages are not found, then the following error message is showing:
External package: ERROR - ERROR: Couldn't find jdbc drivers in /my-extra-packages - ERROR: Looked for: mysql-connector-java-*.tar.gz mysql-connector-java-*.jar mariadb-java-client-*.jar - ERROR: Couldn't find database server package in /my-extra-packages - ERROR: Looked for: mysql-*-linux-glibc2.12-x86_64.tar.gz mariadb-*-linux-x86_64.tar.gz
Trust chain certificates
This check allows you to add root and intermediate certificates to validate the trust chain to establish the connection using TLS. Currently, only pem
files are supported.
-- TLS certificate trustchain Add trusted certificates? (y/n) [No]: ##If you answer “No”, certificates will not be added. ##If you answer “Yes”, All the certificates found at the specified location are imported. Wildcards can be used.
Hadoop cluster manager
This check allows you to specify a cluster manager (Ambari or Cloudera Manager) to configure Unravel automatically.
-- Hadoop cluster manager (Ambari or Cloudera Manager only) Configure a cluster manager? (y/n) [No]: ##If you answer “Yes”, you are prompted for the full URL and credentials to test the connection.
Unravel HTTPS
This check allows you to configure and test HTTPS for the unravel UI. This check prompts you for certificate, key, password, and hostname details that are used to access Unravel.
-- TLS configuration
Note
You must provide the certificate and key in PEM format.
The information provided is verified for the following:
If the Key and Certificate match
If the certificate is valid
If the certificate applies for the provided hostname
Unravel UI port
This check allows you to set the Unravel UI port and verify the connectivity.
-- Unravel default port Port number (integer) [3000]: Do you want to test if the port is accessible? (y/n) [Yes]: This will open port 3000 and listen for connection for 120 seconds. Use your browser to test if the Unravel UI will be accessible on that port. We have detected the following hostnames: - some.host.example Browse to: http://some.host.example:3000 ATTENTION: This address is an example. You should test with the URL that will be used to access Unravel.
A connection on port 3000 is tried and established. If the connection is successful, Unravel Port Test: OK
is shown on the browser, and Unravel port: Reached
is shown on the server.
Unravel directories
This check allows you to set a custom data directory and verify the access if the directories exist. You will always find the software location where you deploy the Unravel binaries. In this check, only the space and access are tested. That data location that you have configured will be used.
-- Unravel directories Software [/opt/unravel]: Data [/opt/unravel/data]: Directories: ERROR - OK: 33 GB of free disk space for software. - ERROR: SYSH0026: Space for data 33 GB is low, recommended minimum is 100 GB.
Email configuration
This check allows you to configure and test email. You are prompted for host and credentials, and the following items are tested:
Connectivity
Authentication, only if provided.
Optional: Send test mail.
-- Mail server (SMTP) configuration
Full precheck
This option allows you to run the full precheck using some of the provided information. Additional tests like user limits, CPU, and memory are run this way.
-- Full precheck
Run Interactive Precheck automatically with existing answers
This can be used to quickly verify previously answered questions (For example, test X failed, was fixed, and re-run everything to ensure it works). Moreover, in the context of automation, you can generate an answer file and run an interactive precheck to verify everything and get a bootstrap configuration.
To obtain the default answers to all the questions:
precheck.sh --show-defaults
To run without user interaction using existing answers:
precheck.sh --headless