Skip to main content

Home

Upgrading sensors on HDP

unravel-host is the Unravel Server's fully qualified domain name or IP address.

spark-version is the target Spark version. For example, 1.6.0, 2.0.1, 2.2.0.

hive-version is the target Hive version. For example, 1.2.0.

Notice

  • Only activate the sensors when no Hive or Spark jobs are running, as the restart may affect running jobs.

  • In a multi-cluster deployment, it is mandatory to upgrade the core node for upgrading to a newer version of sensors on individual clusters.

  1. On Unravel Server and on all nodes in the cluster that have Unravel Sensors deployed, create backups of the old sensor folders and then remove the folders.

    cd /usr/local/
sudo tar -cvf unravel-sensors-`date +%m%d%y`.tar unravel-agent unravel_client
sudo rm -rf /usr/local/unravel-agent
sudo rm -rf /usr/local/unravel_client

  2. Create a directory, for example, /usr/local/unravel-jars, for the JARs on the Unravel node in a cluster. In the case of multi-cluster, create this directory for the jars on the edge nodes. 

    mkdir /usr/local/unravel-jars
chown -R unravel:unravel /usr/local/unravel-jars

  3. Generate the JARs using the unravel_hdp_setup.py script.

    <Unravel installation directory>/unravel/manager run script unravel_hdp_setup.py --sensor-only --unravel-server <unravel-host>:3000 --spark-version <spark-version> --hive-version <hive-version> --ambari-server <ambari-host> --btrace-dir /usr/local/unravel-jars --hive-hook-dir /usr/local/unravel-jars

    Replace the values for unravel-host, spark-version, hive-version, and ambari-host with appropriate values.

    Tip

    For unravel-host, specify the protocol (HTTP or HTTPS) and use the fully qualified domain name (FQDN) or IP address of Unravel Server. For example, https://playground3.unraveldata.com:3000.

  4. Distribute /usr/local/unravel-jars to all worker, edge, and master nodes that run the queries.

    For example,

    scp -r /usr/local/unravel-jars root@hostname:/usr/local/unravel-jars

  5. Make sure the node can reach port 4043 of Unravel Server.

In this section: