Skip to main content

Home

v4.6.1.9 Release notes

Software version

Release Date: June 1, 2021

See v4.6.1.9 for download information.

Software upgrade support

In Unravel v4.6.1.9, the following path is supported for upgrade:

  • 4.6.1.6 to 4.6.1.9

  • 4.6.1.8 to 4.6.1.9

Sensor upgrade

  • A sensor upgrade is mandatory.

Certified platforms

The following platforms are tested and certified for Unravel v4.6.1.9

  • Cloudera Distribution of Apache Hadoop (CDH)

  • Hortonworks Data Platform (HDP)

  • Amazon Elastic MapReduce (EMR)

Review your platform's compatibility matrix before you upgrade or install.

Updates to Unravel's configuration properties

Installation and Upgrade instructions

Refer to Upgrading Unravel server for instructions to upgrade Unravel.

New features

  • None

Improvements and enhancements

  • UI

    • UI changes to show diagnostics message as tooltip/popup for failed task attempts. This is only for Tez. (ASP-1082, ASP-1030)

Bug fixes

  • Kafka

    • NPE exception is seen while processing stream metadata. (ASP-1079)

  • Applications

    • The streaming graph is absent in the last attempt for multi-attempt streaming applications. (ASP-1081)

  • Reports

    • Handle Resource Manager HA for Migration reports. (REPORT-1829)

  • Tez

    • Disable ATS polling on the cloud to avoid data inconsistency and Tez DAG issues. (CUSTOMER-1688)

    • Tez application missing. (SUPPORT-473)

  • UI

    • Unexpected token in index.js. (UX-637)

    • The Spark Streaming app topic is linked to the Kafka page. (CUSTOMER-1705)

Unsupported

  • Hive versions before 1.10 are no longer supported.

Known issues

Application

  • Spark: Lag in reflecting the status of apps in ES. (ASP-974)

AutoAction

  • EMR: Hive metrics may not be published in the RUNNING state. (HIVE-135)

Data Page

  • The accessed partition does not support Hive on Tez etc. It only supports Hive on MR. (DATAPAGE-250)

  • Datapage does not load the total table and partition size KPI's (DATAPAGE-379)

Metrics Collection

  • Resource metrics will not be collected for JDK versions that are equal to or greater than 9 without using the extra JVM switches as mentioned below:

    Workaround: All metrics can be collected if the following options are used for sensor JVM

    --add-exports java.base/jdk.internal.perf=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED

    For example:

    JDK9_MODULE="--add-exports java.base/jdk.internal.perf=ALL-UNNAMED --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED"
ENABLED_SENSOR_FOR_DRIVER="spark.driver.extraJavaOptions=$JDK9_MODULE -Dcom.unraveldata.client.rest.request.timeout.ms=500 -Dcom.unraveldata.client.rest.conn.timeout.ms=500 -Dcom.unraveldata.client.rest.queue=2000 -javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=spark-2.3,config=driver,stdout=true,debug=true"
ENABLED_SENSOR_FOR_EXECUTOR="spark.executor.extraJavaOptions=$JDK9_MODULE -javaagent:unravel-agent-pack-bin.zip/btrace-agent.jar=libs=spark-2.3,config=executor"

    These JVM options are valid for JDK9 - JDK15. Without these JVM options, the following metrics are not collected:

    • ProcessCpuLoad, ProcessCpuTime, SystemCpuLoad

    • PerfCounters metrics

Platforms

  • gc load metric sensor for MR application will not load on EMR.

  • Spark applications that run in the local mode do not use YARN resources, so these applications will be shown using 0 resources in Unravel. (PLATFORM-2809)

  • Intermittent issue in fetching the Btrace data for MR applications. (PLATFORM-2807)

  • Killed MR apps do not display RM diagnostics. (PLATFORM-2815)

Reports

  • Certain users and queues are missing in the Cluster Compare report. (REPORT-1342)

  • Ondemand reports are not generated after the upgrade. (REPORT-1850)

    Workaround: After upgrade, if the reports are not generated, restart the ondemand daemon. 

    sudo service unravel_ondemand restart

  • In the Cluster Compare report, the same value trends are highlighted instead of trends with different values. This is fixed in the New UX. (REPORT-1479)

  • Summary of MapSlotDuration and ReduceSlotDuration is not getting updated in the TopX report. (REPORT-1757)

Spark

  • There is a lag seen while streaming Spark SQL applications. (PLATFORM-2764)

Spark Insights

  • For PySpark applications, the processCPUTime and the processCPULoad are not captured properly. (USPARK-626)

Workflow

  • The number of TEZ applications in an Oozie workflow is sometimes incorrect. (PLATFORM-2403)

UI

  • For streaming applications, the global search with application ID does not work. (UIX-3312)

For support issues, contact Unravel Support.

In this section: