v4.7.9.1 Release notes

Software version

Release date: 03/July/2023

See 4.7.9.1 for download information.

Software upgrade support

The following upgrade paths are supported:

4.7.x.x → 4.7.9.1
4.7.5.0 Hot fix → 4.7.9.1
4.6.1.9 → 4.7.9.1
4.6.1.8 or earlier → 4.6.1.9 → 4.7.9.1

For instructions to upgrade to Unravel v4.6.1.9, see Upgrading Unravel server.

For instructions to upgrade to Unravel v4.7.9.x, see Upgrading Unravel.

For fresh installations, see Installing Unravel.

Sensor upgrade

Sensor upgrade is mandatory for on-prem platforms when you upgrade to Unravel 4.7.9.1. Refer to Upgrading Sensors

Certified platforms

The following platforms are tested and certified in this release:

On-premise platforms

Cloudera Distribution of Apache Hadoop (CDH)
Cloudera Data Platform (CDP)
Hortonworks Data Platform (HDP)

Review your platform's compatibility matrix before you install Unravel.

Updates to Unravel's configuration properties

See 4.7.x - Updates to Unravel properties.

Updates to upgrading Unravel to v4.7.9.1

Obtain License file

An existing license for any previous version does not work with the newer version of Unravel. Therefore, before upgrading Unravel, you must obtain a license file from Unravel Customer Support. For information about setting the license, see Upgrading Unravel from version 4.7.x to 4.7.9.x section in Upgrading Unravel.

Re-configure the custom ports after upgrading from versions before v4.6.1.9

In the case of an Unravel version before v4.6.1.9, you have a customized UI port, for example, 4000, then after the upgrade, this port reverts to the default 3000. Hence, after you upgrade to 4.7.9.1, you must reconfigure the custom port.

Stop Unravel.

<Unravel installation directory>/unravel/manager stop

Display the list of ports and keys.

<Unravel installation directory>/unravel/manager config ports show

Set the port using the following command with the port and port key.

<Unravel installation directory>/unravel/manager config ports set <port key> <port>

For example, if you run the following command, NGUI listens on port 1234.

<Unravel installation directory>/unravel/manager config ports set /hosts/host_main/instances/ngui_1/config/network/port 1234

Note

Run manager config ports unset <port key> command to return the ports to their default values.

<Unravel installation directory>/unravel/manager config ports unset /hosts/host_main/instances/appstore_1/config/flask/port

Apply the changes.

<Unravel installation directory>/unravel/manager config apply

Start Unravel.

<Unravel installation directory>/unravel/manager start

Unset/set properties on the HDP to CDP migrated cluster

After you have migrated from an Hortonworks Data Platform (HDP) to a Cloudera Data Platform (CDP) cluster, there are some properties you must unset and new properties that you must set on an edge node of the migrated cluster.

Single-cluster (HDP to CDP migration)

On the edge node, set the following property:

<Unravel installation directory>/unravel/manager config properties set com.unraveldata.cluster.type CDP

In case HBase was configured earlier, then set the following property:

<Unravel installation directory>/unravel/manager config properties set com.unraveldata.hbase.source.type=CDP

Unset the following properties:

<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.url
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.username
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.password
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.yarn.timeline-service.webapp.address
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.yarn.timeline-service.port

Run manager config auto command to pull in all the Hadoop configurations automatically. You will be prompted to provide the location and credentials for the CDP Cloudera manager URL.
```
<Unravel installation directory>/unravel/manager config auto
```
If there are more than one clusters that are handled by the CDP Cloudera manager, then you are prompted to enable the cluster that you want to monitor. Run the following command to enable a cluster:
```
<Unravel installation directory>/unravel/manager config cluster enable <CLUSTER KEY>
```
Example: /opt/unravel/manager config cluster enable cluster1

Apply the changes.

<Unravel installation directory>/unravel/manager config apply

Start Unravel.

<Unravel installation directory>/unravel/manager start

Ensure to make the following changes in the Cloudera manager:
- HDP/CDH has hive hooks under HIVE service, but for CDP, it is under HIVE_ON_TEZ. Hence, you must update the properties accordingly.
- Update CDP Version for sensor parcel in Parcel Repository & Network from Cloudera Manager to https://xxx.unraveldata.com:3000/parcels/cdh7.1/
- Check for new parcels and distribute them.

Multi-cluster (HDP to CDP migration)

In case of a multi-cluster environment, do the following:

Unset the following properties on the core node:

<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.cluster.type
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.url
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.username
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.password
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.<EDGE KEY>.url
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.<EDGE KEY>.username
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.<EDGE KEY>.password
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.ambari.manager.list

Tip

Run the following commands to obtain the <EDGE KEY>.

<unravel_installation_directory>/unravel/config edge show

Run the following commands to obtain the <CLUSTER KEY>.

<unravel_installation_directory>/unravel/manager support show cluster_access_id/unravel/config edge show

Note

For a multi-cluster environment, if your cluster name changes after the migration from HDP to CDP, you must unset the following properties on the core node:

<Unravel installation directory>/unravel/manager config properties unset javax.jdo.option.<EDGE KEY>_<CLUSTER KEY>_HIVE.ConnectionURL
<Unravel installation directory>/unravel/manager config properties unset javax.jdo.option.<EDGE KEY>_<CLUSTER KEY>_HIVE.ConnectionDriverName
<Unravel installation directory>/unravel/manager config properties unset javax.jdo.option.<EDGE KEY>_<CLUSTER KEY>_HIVE.ConnectionUserName
<Unravel installation directory>/unravel/manager config properties unset javax.jdo.option.<EDGE KEY>_<CLUSTER KEY>_HIVE.ConnectionPassword
<Unravel installation directory>/unravel/manager config properties unset hive.metastore.<EDGE KEY>_<CLUSTER KEY>_HIVE.cluster.ids
<Unravel installation directory>/unravel/manager config properties unset hive.metastore.<EDGE KEY>_<CLUSTER KEY>_HIVE.cluster.ids
<Unravel installation directory>/unravel/manager config properties unset com.unraveldata.hive.metastore.list

Run the following command:

<Unravel installation directory>/unravel/manager config edge auto <EDGE KEY>

Apply the changes.

<Unravel installation directory>/unravel/manager config apply

Start Unravel.

<Unravel installation directory>/unravel/manager start

New features

No new features have been added in this release of Unravel.

Improvements and enhancements

Performance improvements
- Optimized configuration handling
- Improvements in concurrent statistics updates
Data page scaling
- Data page performs efficiently with extensive scaling. Scale performance tested using Hive metastore with 2000000 tables and 17000000 partitions.
- Configuration properties introduced to modify the batch size and set the polling limits for table metadata and partitions metadata.
Impala and YARN cluster identification from cluster ID
The cluster type is included with the cluster ID, which helps in identifying the Impala and YARN clusters. (PLATFORM-3003)
FSImage
- A new configuration property unravel.python.fsimage.java.home has been added to set JAVA home for FSImage processing. ( SUPPORT-2015)
UI enhancements
Implemented consistent usage of UI elements such as cluster ID, tooltips, download CSV option, and so on. (PLATFOM-3003)
Other enhancements
- A new configuration property com.unraveldata.metrics.factor has been added to skip metrics updates for individual applications using a configured metrics factor as done on the sensor side. Refer to 4791 Updates to Unravel properties for more details. ( PIPELINE-1868)

Bug fixes

Installation
- After changing the default value of the host_monitor port, the Monitoring page (Manage > Monitoring) does not load. (INSTALL-3193)
Hive on Spark
For Hive on Spark, the Stage timeline view on the Hive page is not showing any data. (UIX-6149)
UI
On the Clusters > Workload page, after modifying the Display items per page setting from 10 to 50, selecting any insight (such as Policy Violation) redirects to the default setting of 10 items per page. (UIX-6033).

Unsupported

Platforms

Data

In GCP - BigQuery, for the Data page, a count of more than 100 projects is not supported.

BigQuery pricing

For BigQuery pricing, Unravel only supports On-demand analysis pricing. Flat-rate analysis pricing and Storage pricing (Active and Long Term storage) is not supported.

Known issues

Applications

AutoActions

AutoActions stop responding due to an invalid or unsupported HTTP URL or webhook. (AA-575)

BigQuery

On the Application details page, the original query link is missing for some cached queries due to the parallel processing of original and cached queries. (BIGQ-61)
Issue: Sometimes, when you process a large number of BigQuery projects with the manager config bigquery integrate command, you may see the following error:
Provider produced inconsistent result after apply
Workaround: Wait for a few minutes and re-run the command. (INSTALL-2860, INSTALL-2934)

Data page

If tables are created with the same name and are accessed, deleted, and re-created, and if those tables are re-accessed, then their query and app count do not match. (DATAPAGE-502)
The query to fetch tableDailyKPIs is getting timed out in case of a huge table partition (27 million). (DATAPAGE-740)
For Hive metastore 3.1.0 or earlier versions, the creation time of partitions is not captured if a partition is created dynamically. Therefore, the Last Day KPI for the partition section is not shown in Unravel. (DATAPAGE-473)
On the Data page, size data is missing for certain tables in databases, although the partition size is correctly displayed in the Partition Detail section. (DATAPAGE-695)

Installation

Issue: You can encounter a NoIndexFound exception for fresh installations of Unravel on GCP-BigQuery. (BIGQ-104)
Workaround: Run the following CURL command on the Unravel node after the installation.
```
curl -XPUT http://localhost:4171/app-19700101_07
```

Kerberos

Kerberos can only be disabled manually from the unravel.yamlfile.
```
 kerberos:
      enabled: False
```

Reports

Upgrade

After upgrading to v4.7.1.0, Notebooks do not work. You can configure them separately. (REPORT-1895)

Support

For support issues, contact Unravel Support.

In this section:

Home