Skip to main content

Home

Configuring Unravel to monitor multiple clusters

Multi-cluster support allows you to see all your clusters through a "single pane of glass". You need only manage one Unravel instance to connect to and analyze multiple clusters. For example, you can use one Unravel instance to monitor Test, UAT (User Acceptance Test), and Production clusters.

Note

Multi-cluster support is only available for fresh installs.

Basic Configuration
arch-config.png
Unravel core to Unravel edge node configuration to cluster

An Unravel Edge Node needs to be configured for each cluster you are monitoring.

MC-Configuration.png
Prerequisites

The Unravel edge node does not have to be a dedicated node. It is a lightweight installation with one child process.

See the installation instructions for your platform for the prerequisites of the Unravel core node.

Unravel core Installation

  1. Install Unravel v4.6.0.0 on your server for your particular platform.

  2. In /usr/local/unravel/etc/unravel.properties set com.unraveldata.multi-cluster.enable=true.

  3. Restart Unravel daemons.

    /etc/init.d/unravel_all.sh start
Installation for Unravel edge node

Important

For each edge node, you must perform the following steps.

  1. Create a directory for the unravel files.

    mkdir /usr/local/unravel

  2. cd a directory and create unravel.yaml.

    cd /usr/local/unravel
vi /home/unravel/unravel.yaml

  3. Configure Unravel Core Node and the Unravel edge node.

    hosts:
  host_master:
    name: "Unravel Core Node"
    hostname: "core_node_fully_qualified_domain_name"
    roles:
      - core
      - mysql_server
  edge_#:
    name: "Unravel Edge Node #"
    hostname: "edge_node_fully_qualified_domain_name"
    roles:
      - cluster_access

  4. Change the owner of unravel.yaml to unravel.

    chown unravel /home/unravel/unravel.yaml

  5. Download the package and unpack it in the directory you created.

    curl http://caballo.unraveldata.com:8081/artifactory/dist-internal/unravel/release/4.6.0.0/unravel-4.6.0.0.2.tar.gz | tar xz -C /usr/local

  6. Change the owner and group of /usr/local/unravel and its contents to unravel.

  7. Run the setup script.

    Important

    The setup script must be run as $unravel-user.

    chown -R unravel:unravel /usr/local/unravel
/usr/local/unravel/versions/4.6.0.0.2/setup.sh --config=/home/unravel/unravel.yaml

2019-12-05 14:25:20 Instanciating templates ...................................................................... Ok

    If run the script as root or another user the script exits. For example,

    [root@unravelEdgeCluster]# /usr/local/unravel/versions/4.6.0.0.2/setup.sh --config=/home/unravel/unravel.yaml

2019-12-11 14:24:57 FATAL *** /usr/local/unravel/versions/4.6.0.0.2/setup.sh was run as root. This is not supported. Exiting

  8. Run manager.sh to configure Unravel. This script sets up the cluster access and lr proxy related cfgs.

    /usr/local/unravel/services/manager.sh configA

  9. Review the /usr/local/unravel/data/conf/unravel.properties. The following properties should be configured as below

    cluster_access.host=unravel edge node
cluster_access.port=4120
cluster_access.external.host=unravel edge node
cluster_access.external.port=4120
sregistry.mode=proxy
com.unraveldata.lrproxy=true
remote_lr.host=unravel core node
remote_lr.port=4043

  10. Start Unravel on the edge node.

    /usr/local/unravel/services/manager.sh start

  11. Configure the edge node Unravel properties. You must set the following properties

    1. Set these properties as below.

      // set the host port
unravel.server.hostport=unravel edge node

// Configure Spark event logs, mapreduce done directory, yarn aggregated logs directory  cluster type cfgs 
com.unraveldata.cluster.type=CDH
com.unraveldata.job.collector.done.log.base=/user/history/done
com.unraveldata.spark.eventlog.location=hdfs:///user/spark/applicationHistory
com.unraveldata.job.collector.log.aggregation.base=/tmp/logs/*/logs/

    2. If you are running Impala you set the following properties.

      com.unraveldata.cloudera.manager.url=http://cm-host:port
com.unraveldata.cloudera.manager.username=userName
com.unraveldata.cloudera.manager.password=PassWord

    3. See basic log properties for all the log properties you need to set.

In this section: