Skip to main content

Home

Configure YARN-MapReduce JVM sensor cluster-wide

  1. In Cloudera Manager, go to YARN service.

  2. Select the Configuration tab.

  3. Search for Application Master Java Opts Base and concatenate the following XML block properties snippet (ensure to start with space and add below).

    Note

    Make sure that "-" is a minus sign. You need to modify the value of unravel-host with your Unravel Server IP address or a fully qualified DNS. For multi-host Unravel installation, use the IP address of Host2.

    -javaagent:/opt/cloudera/parcels/UNRAVEL_SENSOR/lib/java/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043

  4. Search for MapReduce Client Advanced Configuration Snippet (Safety Valve) for mapred-site.xml in the middle of the page.

  5. Enter the following XML four-block properties snippet to Gateway Default Group. (Click View as XML.)

    <property>
<name>mapreduce.task.profile</name>
<value>true</value>
</property> 
<property>
<name>mapreduce.task.profile.maps</name>
<value>0-5</value>
</property> 
<property>
<name>mapreduce.task.profile.reduces</name>
<value>0-5</value>
</property> 
<property>
<name>mapreduce.task.profile.params</name>
<value>-javaagent:/opt/cloudera/parcels/UNRAVEL_SENSOR/lib/java/btrace-agent.jar=libs=mr -Dunravel.server.hostport=<unravel-host>:4043</value></property>

    Note

    Replace unravel-host to Unravel hostname. In case of multi-cluster, add the hostname of the edge node.

    Note

    If you are using JDK version 9 or later, ensure to add the following to the existing JAVA options:

    --add-exports java.base/jdk.internal.perf=ALL-UNNAMED 
--add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
--add-exports java.management/sun.management.counter.perf=ALL-UNNAMED
--add-exports java.management/sun.management.counter=ALL-UNNAMED

    For example, the complete JAVA options are specified as follows:

    -javaagent:/opt/cloudera/parcels/UNRAVEL_SENSOR/lib/java/btrace-agent.jar=libs=mr -Dunravel.server.hostport=unravel-host:4043 --add-exports java.base/jdk.internal.perf=ALL-UNNAMED  --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED --add-exports java.management/sun.management.counter.perf=ALL-UNNAMED --add-exports java.management/sun.management.counter=ALL-UNNAMED

  6. Save the changes.

  7. Deploy the client configuration by using the Actions pull-down menu.

    Cloudera Manager will specify a restart which is not necessary to effect these changes. (Click Restart Stale Services if that is visible. However, you can also perform this later when you have planned maintenance.)

Tip

The restart is important for the MR sensor to be picked up by queries submitted via Hiveserver2.

Use the Unravel UI to monitor the situation. When you view the MapReduce APM page for any completed MRjob you should see mappers and reducers in the Resource Usage tab.

In this section: