unravel.audit.enabled

True - Audit is enabled for Unravel. The Audit tab is displayed in the UI for admin users.

False - Audit feature is disabled and the Audit tab is hidden in the UI.

True

unravel.audit.rest.timeout.seconds

The time period in number of seconds when an event can be added to the database after each invocation. Events that cannot be added within this time will be dropped. The timeout is set to prevent long waits that would slow down Unravel.

2

unravel.audit.dateFormat

Date and time format for Audit Event logs.

Default is YYYY-MM-DDTHH:mm:ss.SSS[Z]

Alerts

com.unraveldata.monitoring.keytab.check.interval

Keytab query interval in minutes.

3 minutes

com.unraveldata.monitor.keystore.expire.days

If certificate expiry falls within this number, an alert is sent.

-

com.unraveldata.monitor.truststore.expire.days

If certificate expiry falls within this number, an alert is sent.

-

com.unraveldata.monitoring.keystore.check.interval

Keystore query interval in minutes.

5 minutes

SAML

com.unraveldata.saml.groupFilter

Comma-separated list of SAML Group names (short name not full DNs).

-

SSL/TLS

com.unraveldata.http.client.ssl.key_store_path

true: SSL enabled for cluster connectors.

false

com.unraveldata.http.client.ssl.key_store_password

Key store password as plain text.

-

com.unraveldata.datastore.port.https

Datastore host.

127.0.0.1

datastore.ssl.enabled

if true, enables SSL for Datastore.

false

com.unraveldata.server.ssl.trust_store_path

Trust store file path.

-

com.unraveldata.server.ssl.trust_store_password

Trust store password in plain text.

-

SSL/TLS

com.unraveldata.cluster_access.grpc_client.ssl.trust_store_path

com.unraveldata.http.client.ssl.trust_store_path

com.unraveldata.cluster_access.grpc_client.ssl.trust_store_password

com.unraveldata.http.client.ssl.trust_store_password

com.unraveldata.cluster_access.grpc_client.log_network_activity

com.unraveldata.log_network_activity

com.unraveldata.server.log_network_activity

com.unraveldata.log_network_activity

com.unraveldata.server.ssl.cert_path

com.unraveldata.server.ssl.key_store_path

com.unraveldata.server.ssl.cert_password

com.unraveldata.server.ssl.key_store_password

com.unraveldata.cloudera.manager.list

Comma-delimited list of variables to designate one or more CM instances.

-

com.unraveldata.cloudera.manager.X.url

Cloudera URL for variable X.

-

com.unraveldata.cloudera.manager.X.username

Cloudera user name for variable X

-

com.unraveldata.cloudera.manager.X.password

Cloudera password for variable X

-

com.unraveldata.cloudera.manager.X.unravel.cluster.ids

Cloudera manager UUID and display name.

-

Ambari

com.unraveldata.ambari.manager.list

Comma-delimited list of variables to designate one or more Ambari instances.

This is used for configuration purposes only and does not need to correspond to Ambari hostnames or cluster ID/name.

-

com.unraveldata.ambari.X.url

Ambari URL for variable X.

-

com.unraveldata.ambari.X.username

Ambari user name for variable X

-

com.unraveldata.ambari.X.password

Ambari password for variable X

-

Hive Metastore

com.unraveldata.hive.metastore.list

Comma-delimited list of variables to designate one or more Hive metastore instances.

-

com.unraveldata.cloudera.manager.X.unravel.cluster.ids

Replace X in the property with one of the metastore variables listed in com.unraveldata.hive.metastore.list

The value is a comma-delimited list of Unravel cluster IDs, and in the case of Impala, the Cloudera Manager (CM) cluster IDs.

-

javax.jdo.option.X.ConnectionURL

Hive metastore URL for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionDriverName

Hive metastore JDBC driver name for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionUserName

Hive metastore JDBC user name for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionPassword

Hive metastore JDBC password for variable X listed in com.unraveldata.hive.metastore.list

-

Kafka

com.unraveldata.ext.kafka.cluster.list

Cluster list. These user-defined names are used to clearly identify the Kafka cluster in the Unravel UI. Use a comma-separated list for multiple clusters.

-

Reports

unravel.python.reporting.cluster.unravel.id

Specify the cluster UID which is used to generate the following reports:

unravel.server.protocol

Value can be http or https based on whether Unravel UI URL is http or https. Default is http.

Used to retrieve an API token so it can access the heatmap and app types information used in the Workload Fit analysis.

http

Pipelines

com.unravel.workflows.compareupdate.disable

-

Multi-cluster configurations for components accessed from Core node server

com.unraveldata.cloudera.manager.list

Comma-delimited list of variables to designate one or more CM instances.

-

com.unraveldata.cloudera.manager.X.url

Cloudera URL for variable X.

-

com.unraveldata.cloudera.manager.X.username

Cloudera user name for variable X

-

com.unraveldata.cloudera.manager.X.password

Cloudera password for variable X

-

com.unraveldata.cloudera.manager.X.unravel.cluster.ids

Cloudera manager UUID and display name.

-

Ambari

com.unraveldata.ambari.manager.list

Comma-delimited list of variables to designate one or more Ambari instances.

This is used for configuration purposes only and does not need to correspond to Ambari hostnames or cluster ID/name.

-

com.unraveldata.ambari.X.url

Ambari URL for variable X.

-

com.unraveldata.ambari.X.username

Ambari user name for variable X

-

com.unraveldata.ambari.X.password

Ambari password for variable X

-

Hive Metastore

com.unraveldata.hive.metastore.list

Comma-delimited list of variables to designate one or more Hive metastore instances.

-

com.unraveldata.cloudera.manager.X.unravel.cluster.ids

Replace X in the property with one of the metastore variables listed in com.unraveldata.hive.metastore.list

The value is a comma-delimited list of Unravel cluster IDs, and in the case of Impala, the Cloudera Manager (CM) cluster IDs.

-

javax.jdo.option.X.ConnectionURL

Hive metastore URL for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionDriverName

Hive metastore JDBC driver name for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionUserName

Hive metastore JDBC user name for variable X listed in com.unraveldata.hive.metastore.list

-

javax.jdo.option.X.ConnectionPassword

Hive metastore JDBC password for variable X listed in com.unraveldata.hive.metastore.list

-

Kafka

com.unraveldata.ext.kafka.cluster.list

Cluster list. These user-defined names are used to clearly identify the Kafka cluster in the Unravel UI. Use a comma separated list for multiple clusters.

-

com.unraveldata.ext.kafka.X.bootstrap_servers

Comma-separated list of kafka brokers for that cluster.

-

com.unraveldata.ext.kafka.<server>.jmx_servers

Comma-separated list of jmx  servers

-

com.unraveldata.ext.kafka.<server>.jmx.kafka1.host

Host name of jmx server.

-

com.unraveldata.ext.kafka.<server>.jmx.kafka1.port

Port of jmx server.

-

com.unraveldata.ext.kafka.<server>.jmx.kafka1.run_period

Collection interval for jmx metrics.

-

Pipelines

com.unravel.workflows.compareupdate.disable

Suppresses redundant message processing for workflows.

com.unraveldata.min.job.duration.for.attempt.log

Minimum duration of a successful application or which executor logs are processed (in milliseconds).

600000

(10 mins)

com.unraveldata.job.collector.log.aggregation.base

HDFS path to the aggregated container logs (logs to process). Don't include the hdfs:// prefix. The log format defaults to TFile. You can specify multiple logs and log formats (TFile or IndexedFormat).

Example: TFile:/tmp/logs/*/logs/, IndexedFormat:/tmp/logs/*/logs-ifile/.

For HDP set this to: IndexedFormat:/app-logs/*/logs/.

/tmp/logs/*/logs/

com.unraveldata.job.collector.done.log.base

HDFS path to "done" directory of MR logs. Do not include the hdfs:// prefix.

For HDP set this to: /mr-history/done.

hdfs:///user/spark/applicationHistory/

com.unraveldata.spark.eventlog.location

All the possible locations of the event log files. Multiple locations are supported as a comma-separated list of values.

This property is used only when the Unravel sensor is not enabled. When the sensor is enabled, the event log path is taken from the application configuration at runtime.

hdfs:///user/spark/applicationHistory/

com.unraveldata.cloudera.manager.api.version

Returns the clusters managed by the Cloudera Manager (CM), when the default CM API version fails to do so.

CM API version is provided by the CM via API: http|https://<CM_HOST>:<CM_PORT>/api/version.

null

com.unraveldata.login.aad.config

Specify the path to the aad.json file

/usr/local/unravel/etc/aad.json

Billing service

unravel.billing.enabled

Set this to true to enable the billing service.

false

unravel.billing.pricingplan

Set this to any of the following plans:

-

unravel.billing.startdate.plan

Specify the start date of the plan in yyyy-mm-dd format. For example: 2021-01-01. The start date must be the first day of the month.

When you add more credit, you must specify a new start date via this property. The new credit takes effect on the new start date. The earliest start date is the month after the current month.

-

unravel.billing.enddate.plan

Specify the end date of the plan in yyyy-mm-dd format. For example: 2021-01-31.

The end date should be the last day of the month and a date after the start date.

-

unravel.billing.unit

Set this to any of the following:

For dbu, the cost of a cluster is attributed to the day when the cluster terminates.

When dbu is selected, users need to create a pricing config file that specifies different prices for different workload types.

-

unravel.billing.defaultprice

When unravel.billing.unit=dbu, this property (unravel.billing.defaultprice) specifies the default price in USD for any workload type that is not listed in the pricing config file.

When unravel.billing.unit=instance_hours, this property specifies the price in USD for all instance types.

When this property is not specified, the default value is 0.1 ($0.10).

You must specify the value with at least one integer and one or two fractional digits. For example, 0.22 or 1.0.

0.1 ($0.10)

unravel.billing.startingcredit

Only applicable when the plan is unravel.billing.pricingplan=pay_in_advance.

This property specifies the amount of money in USD that the customer pays in advance.

The credits are deducted at the end of each day by the amount that corresponds to the usage (measured in unravel.billing.unit) multiplied by the price per unit.

In the case of DBU, there can be a different price for each workload type, as specified in the pricing config file.

You must specify the value with at least one integer and one or two fractional digits. For example, 0.22 or 1.0.

-

unravel.billing.instance.hour.price

Price of per-instance/node hour in the set currency. For example, dollars.

0.10

unravel.billing.dayofmonth.report

A day in a month to generate and email the monthly usage bill.

1

unravel.billing.currency

Currency code/symbol used for billing.

$

unravel.python.reporting.cluster.name

Specify the cluster name which is used to generate the following reports:

In case this property is not specified, then com.unraveldata.cluster.name property is used.

If even that property is not specified, the cluster list in the local system is used and the reports are generated for the first cluster in the list.

-

com.unraveldata.report.excluded.queues

The queues to exclude from metric collection and the queue analysis report.

-

com.unraveldata.migrationplanning.workloadfit.max.apps

Configure the limit on maximum number of apps shown in a single slice of workload fit report and heatmap generation.

1 million

com.unraveldata.migrationplanning.workloadfit.timeout

Configure the timeout for the workload fit report and heat map generation.

90 s

com.unraveldata.process.executor.log

Set the flag to process the executor logs.

true

com.unraveldata.process.event.log

Processes event logs. The default is set to True. If set to False the event logs are not processed.

true

spark.unravel.program.path

hdfs/S3/adl path to the source zip file of the Spark app. This is a new way to upload spark program. Unravel daemons can access this file.

-

com.unraveldata.client.resolve.hostname

If set to false , the Unravel server hostname is not resolved and the DNS name is used to communicate with Unravel.

true

com.unraveldata.oozie.staleworkflow.fetch.num

The maximum number of stale workflows to update in a single run.

100

com.unraveldata.oozie.staleworkflow.interval.sec

The frequency of the task.

120 s

com.unraveldata.oozie.staleworkflow.fetch.logs

Whether to fetch Oozie logs for a stale workflow.

false

com.unraveldata.oozie.staleworkflow.starttimedelta.min

The minimum time interval that determines if a workflow is stale.

30 min

com.unraveldata.oozie.fetch.logs

Whether to fetch the Oozie logs for a workflow.

true