Skip to main content

Home

v4.7.1.0 Release notes

Software version

Release Date: 12/Jul/2021

See v4.7.1.0 for download information.

Software upgrade support

The following upgrade paths are supported:

  • 4.6.1.8 or earlier → 4.6.1.9 → 4.7.1.0

  • 4.6.1.9 → 4.7.1.0

  • 4.7.0.1 → 4.7.1.0

Sensor upgrade

  • Sensor upgrade is mandatory.

Certified platforms

The following platform is tested and certified in this release:

  • Hortonworks Data Platform (HDP)

  • Cloudera Distribution of Apache Hadoop (CDH)

  • Cloudera Data Platform (CDP)

  • Amazon Elastic MapReduce (EMR)

  • Databricks

Review your platform's compatibility matrix before you install Unravel.

Updates to Unravel's configuration properties

New features

  • Beta support for 4.6.x upgrade to target version 4.7.1.0

    You can now upgrade from Unravel version 4.6.x to 4.7.1.0 using the following upgrade paths:

    • 4.6.1.8 or earlier → 4.6.1.9 → 4.7.1.0

    • 4.6.1.9 → 4.7.1.0

    Notice

    The upgrade must be done only with the help of Unravel Customer Success team.

  • Enhancements for cloud migration

    • PDF downloads

      You can now download all the Migration reports and heatmaps in PDF format.

    • Configure cloud providers for the display

      You can configure the cloud providers of your choice for display in the drop-down for Migration reports. Only the configured cloud providers will be made available for the Migration reports.

    • Restrict display of all the VMs for Migration reports

      Using a CSV or txt file, you can configure a blacklist of VM instances. The blacklisted VM instances are not available for Migration reports. The VM instances can be blacklisted either for all cloud providers or specific cloud providers.

    • Workload fit report

      • Removed the limit on the number of apps shown in a single slice of Workload fit report and heatmap generation.

Improvements and enhancements

  • Databricks

    • Add Alerts panel in Clusters > Overview tab for Databricks. (UIX-3758)

    • You can integrate Unravel with Azure Active Directory (AAD) to use this authentication service of Azure for Databricks from Unravel new UI. (RBAC-17, RBAC-26, RBAC-28,RBAC-35, RBAC-36)

  • Migration

    • Workload Fit: Move heatmap generation logic to ondemand backend. (MIG-7)

    • Backend API for Workload Fit report to list tags. (MIG-14)

  • Install

    • Provide the ability for the manager service to configure FSimage properties. (CUSTOMER-1697)

    • Correctly accept multiple uses of the --pem switch when configuring certificates. (INSTALL-1504)

  • Notebooks

    • Multi-cluster support for Hive, Spark, and Impala workload analysis notebooks. (PLATFORM-2441)

    • Multi cluster support for generic Jupyter notebooks. (REPORT-1853)

  • Tez

    • Add support to process yarn logs for Tez applications. (ASP-1077)

    • TEZ - Diagnostics info for task attempts. (CUSTOMER-1663)

  • UI

    • UI improvements:

      • Upgraded UI Node version to 14.17.0.

      • Navigate to the last visited page after auto-logout.

      • Improved keyboard navigation on Login page, navigation menu items, and chart export menus.

      • Improved trial expiry message for readability.

      • Layout improvements for Cluster pages such as Overview, Resources, Chargeback, Workload, etc.

      • Moved Type dropdown along with other filter items for Cluster pages such as Resources, Chargeback, Job Trends, and Workload.

      • Improved Jobs Applications page to remove the clutter and text consistency.

      • Layout improvements for Data pages; Overview, Tables, Small Files, Forecasting.

      • Added confirmation for Scheduled report deletion.

    • Add class/ID/name for cluster and type selection. (UIX-3910)

    • Chargeback page: Added the ability to change the Group by to any desired order. (CUSTOMER-673)

Unsupported

Billing

  • Billing is not supported on EMR and on-prem platforms.

Data

  • The Data insights feature is not supported for EMR and Databricks clusters.

  • Other than Overview and Tables, the Data page is not supported for EMR and Databricks clusters.

Jobs

  • Impala jobs are not supported on the HDP platform.

Healthcheck

  • Monitoring the expiration of the SSL Certificates and Kerberos principals in Unravel multi-cluster deployments.

Migration Planning

  • Migration Planning is not supported for the following regions for Azure Data Lake:

    • Germany Central (Sovereign)

    • Germany Northeast (Sovereign)

  • Forecasting and Migration: In a multi-cluster environment, only a single cluster can be configured at a time; hence reports are generated only for that single cluster.

Multi-cluster deployment

  • Unravel does not support multi-cluster management of combined on-prem and cloud clusters.

Pipeline

  • Unravel does not support apps belonging to the same pipeline in a multi-cluster environment but is sourced from different clusters. A pipeline can only contain apps that belong to the same cluster.

Reports

  • Reports are not supported on Databricks and EMR.

Sessions

In Jobs > Sessions, the feature of applying recommendations and then running the newly configured app is not supported.

UI

  • Pig and Cascading applications are not supported.

Bug fixes

  • Data page

    • The inconsistencies for Trend graphs view for Users, Apps, and Size data for all tables are fixed under Data page > Tables. (DATAPAGE-482)

  • Install

    • autoconfig: Support added for spark 3. (INSTALL-1674)

  • Kafka

    • Fix the out-of-memory issue for Kafka topics due to JAVA parameters. (INSTALL-1020)

    • Linking Spark Streaming app topic to Kafka page. (UIX-108)

  • Migration

    • Fixed issues associated with tag selection on the Workload Fit page. (MIG-66)

    • Workload Fit report generation fails if there is a single quote in tag value. (MIG-70)

    • Workload Fit scalability issue: workloadfit tags are not loading in a large number of tag values. (CUSTOMER-1654)

  • Reports

    • Workload Fit report: Pie chart checkbox not working as expected. This is fixed. (UIX-3909)

    • Actions > More info not working for Scheduled > Cloud Mapping Per Host report. This is fixed. (UIX-3920)

  • UI

    • Aggregate dropdown is misaligned. (UIX-3912)

    • Vcore/memory values are not the same in graph and listed applications. (CDI-219)

    • Line graphs do not render properly after viewing the Workload page. (UIX-3908)

    • The future date was displayed in the table title for Clusters > Resources and Clusters > Jobs Trends. (CDI-242)

    • Clusters > Chargeback - Application details are missing in downloaded CSV file. (CDI-240)

  • Upgrade

    • After upgrade from v4.6.1.9 to v4.7.1.0, the data is not displayed on the Resource Usage > Impala page. (CDI-229)

    • While upgrading from 4610 to 4701, the Hive on MR applications does not load the MR jobs in the navigation tab. (ASP-1095)

Known issues

Applications

  • Duplicate apps with the same cluster UID entry are found in Elastic Storage. (ASP-1044)

  • In the Tez app, Dag all vertices table KPI is shown as blank if the value is missing or 0. (ASP-1128)

Data page

  • Latest Access column does not display the Date/Time when the table was last accessed in Data page> Tables > Details Table. (DATAPAGE-466)

  • Apps count is not getting displayed for the expected tables in the Table Details page for a selected table. (DATAPAGE-469)

  • Wrong data displayed for Number of Partitions Created KPI/trend graph under Partitions KPIs - Last Day section in Data page. (DATAPAGE-473)

  • Wrong database name displayed for a hive application in App Summary page (DATAPAGE-474)

  • Number of Tables Created KPI/Trend graph does not reflect the tables data for the last day even after restarting the table worker daemon. (DATAPAGE-478)

  • The wrong count is displayed for Total Number of Tables KPI/Trend graph under Data >Overview page. (DATAPAGE-479)

  • Running FSImage leaves some KPIs in an inconsistent state until the table worker daemon is restarted. (DATAPAGE-481)

  • Incorrect data is displayed for Size of Tables Created and Total Size of Tables KPI/trend graphs under Data page > Overview > Tables KPIs > Trends section. (DATAPAGE-483)

  • In Data page > Tables section, the Latest Access column is not updating with the latest accessed timestamp, which causes wrong data to be populated in the Users and Apps trend charts. (DATAPAGE-484)

Kerberos

  • Kerberos can only be disabled manually from the unravel.yamlfile. 

     kerberos:
      enabled: False
Platform

  • com.unraveldata.clusteraccess.hdfsconnector.api.HdfsConnectorException error is shown in the yarn_jc_sensor.log file. (PLATFORM-2956)

Reports

  • Cluster discovery

    • If the metric retrieval for a host fails, then the CPU and memory capacity/usage graphs and heatmaps are not displayed.

      This happens on a CDH cluster when the Cloudera Manager agent of a host does not send any heartbeats to the Cloudera Manager server. Such a host is shown as Bad Health in Cloudera Manager. (REPORT-1706)

      Workaround: Ensure that the Cloudera Manager agent sends heartbeats to the Cloudera Manager on all hosts and that none of the hosts are shown as Bad Health.

    • The On-prem Cluster Identity may show an incorrect Spark version on CDH. The report may incorrectly show Spark 1 when Spark 2 is installed on the CDH cluster. (REPORT-1702)

  • When using PostgreSQL, the % sign is duplicated and displayed in the Workload Fit report > Map to single cluster tab. (MIG-42)

  • Cloud Mapping Per Host report scheduled in v4.6.1.x will not work in v4.7.1.0. Users must schedule a new report. (REPORT-1886)

  • The TopX report email contains a link to the Unravel TopX report instead of showing the report content in the email as in the old reports.

  • Queue analysis: The log file name (unravel_us_1.log) displayed in the error message is incorrect. The correct name of the log file is unravel_sensor.log. (REPORT-1663)

Sensor

  • The sensor setup script fails with unrecognized arguments. (INSTALL-1667)

Sessions

  • Spark Analyse session fails in the absence of event logs for the applications present in the session. (SESS-366)

  • Session creation from the Hive application page is not supported. (UIX-3577)

    Workaround: Use the Sessions listing page to create a hive session.

  • The session should be sorted by create date instead of session names. (SESS-368)

  • Status of Application is seen as UNKNOWN in session details when the application has multiple attempts. (SESS-369)

Spark

  • There is a lag seen for SQL Streaming applications. (PLATFORM-2764)

Security

  • If the customer uses an active directory for Kerberos and the samAccountName and principal do not match, this can cause errors when accessing HDFS.

Spark insights

  • For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)

Tez

  • SQL events generator generates SQL Like clause event if the query contains like pattern even in the literals. (TEZLLAP-349)

  • Dag additional input and output node is not plotted for 4710 Tez applications. (ASP-1114)

Upgrade

  • Notebooks will not work after upgrading to v4.7.1.0. You can configure them separately. (REPORT-1895)

  • After you upgrade from v4.6.x to v4.7.1.0, the Tez application details page does not initially show DAG data. The DAG data is visible only after you refresh the page. (ASP-1126)

UI

  • The new user interface (UI) can be accessed only from Chrome.

  • In the App summary page for Impala, the Query> Operator view is visible after scrolling down. (UIX-3536).

Workflow

  • Jobs getting falsely labeled as a Tez App for Oozie Sqoop and Shell actions. (PLATFORM-2403)

Support

For support issues, contact Unravel Support.

In this section: