4.8.2.0 Release notes
Software version
Release date: November 10, 2023
See 4.8.2.0 for download information.
Software upgrade support
Note
Upgrading is not supported in this release.
For fresh installations, see Installing Unravel.
Certified platforms
The following platforms are tested and certified in this release:
Cloud platforms
Google Cloud Platform (GCP) - BigQuery
Snowflake
Review your platform's compatibility matrix before you install Unravel.
Updates to Unravel's configuration properties
See 4.8.x - Updates to Unravel properties.
New features
Improvements and enhancements
Upgraded OpenSearch and OpenSearch dashboards to 2.9.0 version. (INSTALL-3348)
For enhanced security and permission management, the getData permissions are added to Terraform for BigQuery integration only if the user has opted to enable the Data page or set the Billing data feature. (INSTALL-3414)
The command --create-credentials has been replaced with --need-integration to prevent misinterpretation. (BIGQ-795)
UI and server Node.js are now upgraded to 18.14.0. (UIX-5577, UIX-5578)
Python updated to the latest 3.8.17 version. (INSTALL-3417)
Bug fixes
BigQuery
The following bugfixes are included in this release for Unravel BigQuery:
Insights
Scheduled query insights are not generated for on-demand queries. (BIGQ-702)
The action plan associated with Reservation Insights is missing. (BIGQ-761)
Jobs
All projects that are configured for Unravel monitoring should be displayed under the Projects page even if the queries or jobs are not run for those projects. (BIGQ-449)
When comparing two jobs on the Jobs page, the Creation time is not shown on the Job details page when Show differences is enabled, even though the creation time is different. (BIGQ-615)
Snowflake
The following bugfix is included in this release for Unravel Snowflake:
Warehouses
Unravel (Integration Service) IS queries are preventing the suspension of warehouses in the environment. (SNOW-997)
Sustained Violation is not supported for Databricks AutoAction.
All the reports, except for the TopX report, are not supported on Databricks and EMR.
Snowflake
The following Known issues are included in this release for Unravel Snowflake:
Some of the features on the Data page, such as the trend graphs on the Data Overview page and on the Data Tables page, are not applicable for Snapshot mode in Snowflake. (SNOW-1281)
Data quality insights are not supported for Snapshot mode. (SNOW-1281)
If tables are created with the same name and are accessed, deleted, and re-created, and if those tables are re-accessed, then their query and app count do not match. (DATAPAGE-502)
For Hive metastore 3.1.0 or earlier versions, the creation time of partitions is not captured if a partition is created dynamically. Therefore, the Last Day KPI for the partition section is not shown in Unravel. (DATAPAGE-473)
Workflow/Job page displays empty for Analysis, Resources, Daggraph, and Errors tabs. (DT-1093)
Issue: Jobs created for PySpark application with UDF on a JobCluster fail after applying the recommendations for node downsizing. (DT-1404)
Workaround:
In your Databricks workspace, go to Configure Cluster > Advanced Options > Spark config
Add and set the following property to true for spark.driver.extraJavaOptions and spark.executor.extraJavaOptions spark configurations:
-Dcom.unraveldata.metrics.proctree.enable=true
For example:
spark.executor.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=executor,libs=spark-3.0 spark.driver.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=driver,script=StreamingProbe.btclass,libs=spark-3.0
Data is not displayed when you click the Optimize button corresponding to OTHERS for the Cost > Chargeback results shown in the table. (UIX-5624)
On the Job Runs page, the Cost slider stops working when the cost for the selected user is less than $1. (UIX-5508)
The job run count displayed on the Chargeback page differs from the job count shown on the Workflow page. (UIX-5581)
Errors and Logs data are missing in specific scenarios for a failed Spark application, such as an application failing with
OutOfMemoryError
. (ASP-1624)On the Cost > Trends and Cost > Chargeback pages, the tooltip for the Last <number of> days field includes more days than the displayed days. (UIX-5042)
Clusters with an Unknown status are excluded from the dashboards often used for monitoring systems. (DT-1445)
When the Interactive cluster is restarted, the cluster count is increased on the Databricks Cost > Trends page. (DT-1275)
After navigating from the Trends and Chargeback pages with Tag filters, the
No data available
message is displayed on the Compute page. (DT-1094)Inconsistent data is displayed for the cluster Duration and Start Time on the Compute page. (ASP-1636)
The DriverOOME and ExecutorOOME events are not generated for the Databricks notebook task. (DT-533)
When a job fails to submit a Spark application, the failed DataBricks job is missing from the Unravel UI. (ASP-1427)
In the Databricks view, the application is shown in a running state, even though the corresponding Spark application is marked as finished. (ASP-1436)
On the Workflows > Jobs page, you can view only up to 100 records or jobs. (ASI-695)
The Email template for the AutoAction policy contains the unformatted table. The issue occurs when the threshold value is zero (0). (ASI-688)
Azure Databricks jobs are randomly missing on the Unravel UI due to Azure Databricks File System (DBFS) mount issues. (PIPELINE-1626)
The Job Run details page displays a duplicate entry for tasks executed during the job. (DT-1461)
Issue: Jobs created for PySpark application with UDF on a JobCluster fail after applying the recommendations for node downsizing. (DT-1404)
Workaround:
In your Databricks workspace, go to Configure Cluster > Advanced Options > Spark config
Add and set the following property to true for spark.driver.extraJavaOptions and spark.executor.extraJavaOptions spark configurations:
-Dcom.unraveldata.metrics.proctree.enable=true
For example:
spark.executor.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=executor,libs=spark-3.0 spark.driver.extraJavaOptions -Dcom.unraveldata.metrics.proctree.enable=true -javaagent:/dbfs/databricks/unravel/unravel-agent-pack-bin/btrace-agent.jar=config=driver,script=StreamingProbe.btclass,libs=spark-3.0
An exception occurs when installing Unravel version 4.7.6.0 with the Azure MySQL database (SSL Enabled). (INSTALL-2799)
The Insights Overview tab uses UTC as the timezone, while other pages use local time. Hence, the date and time shown on the Insights Overview tab and the other pages after redirection can differ. (UIX-4176)
Kerberos can only be disabled manually from the
unravel.yaml
file.kerberos: enabled: False
TopX report
The TopX report email links to the Unravel TopX report instead of showing the report content in the email as in the old reports.
If the Spark job is not running for Databricks, the values for the Duration and End time fields are not updated on the Databricks Run Details page. (ASP-1616)
You can see a lag for SQL Streaming applications. (PLATFORM-2764)
On the Spark application details page, the timeline histogram is not generated correctly. (UX-632)
For PySpark applications, the
processCPUTime
andprocessCPULoad
values are not captured properly. (ASP-626)
After upgrading to v4.7.1.0, Notebooks do not work. You can configure them separately. (REPORT-1895)
In the App summary page for Impala, the Query> Operator view is visible after scrolling down. (UIX-3536).
When you click the hive query, which was executed as part of the Hive on Spark application, a blank page is shown. (UIX-6037)
On the Clusters > Resources page, in the Group By drop-down list, the Application Type, User, and Queue options are duplicated for the YARN/IMPALA resource job type. The issue occurs if identical user-defined tags are used. (UIX-5898)
Snowflake
The following limitations are included in this release for Unravel Snowflake: