Skip to main content

Home

Chargeback

You can generate chargeback reports for your clusters' usage costs for Yarn jobs and Impala jobs. Multi-cluster feature is supported for Chargeback reports. However, you can only view the report of a single cluster at a time.

If Yarn jobs or Impala jobs are not running on the selected cluster at the specified time, then chargeback reports are not shown.

Generate chargeback report

  1. From the Chargeback Type drop-down, select either Yarn or Impala.

  2. Form the Cluster dropdown select a specific cluster.

  3. Form date picker pull-down menus select a date range.

  4. In the VCore/Hour ($) , enter the estimated number of VCores used per hour, and in the Memory MB/Hour ($), enter the estimated memory used per hour. A quick estimation is displayed in the results table, in the CPU hours or Memory hours column of the corresponding group by the table.

    group-bytable.png

  5. Click in the Group By box and select an option. Select a maximum of two Group By options at a time. You can click Deselect-X.png next to the option to deselect an option if you have selected more than one option.

    The chargeback report is generated. If you have selected two Group By options, the combined results are displayed in the donut charts (Jobs, CPU hours, Memory hours) and the table below the donut charts. Refer to Drilling down the Chargeback results for more details.

    Example:

    In the following image, the report is grouped by two tags, dept and project. (See What is tagging, if you are unfamiliar with the concept.)

    chargeback.png

    There are times when a job cannot be grouped by the selected option. In such cases, NULL is listed in the Group By option column. In this case, there are jobs that have neither a dept nor project tag.

    Hover over a donut section to see the slice name, the value, and the percentage of the whole.

    The list of all the Yarn jobs/Impala jobs is provided in the tables as shown. You can view 15 records at a time and download the list in a CSV format.

    chargeback-yarn-imp-jobs1.png
    chargeback-yarn-imp-jobs.png

    Click export-format.png to download the chargeback report in CSV format.

Drilling down the Chargeback results

The chargeback results are displayed in the table as shown in the following image:

chargeback-results-table.png

Note

For BigQuery, if parent and child jobs are available, only the parent jobs are listed in the chargeback result because the parent job contains the aggregated metrics of child jobs.

Click any row to drill down to the next Group By option. If you click the row again, you can drill down further till all the associated Group By options are exhausted for the Application type option in Yarn jobs and User option in Impala jobs.

Use the navigation path corresponding to the Group by to move back to a previous option.

Estimating chargeback cost

  1. From the Chargeback drop-down in the upper right corner, select one of the following options:

    • Yarn

    • Impala

  2. Use the Cluster and date picker pull-down menus to select a specific cluster or change the date range.

  3. Click in the Group By box and select an option. You can select a maximum of two Group By options only at a time. The Chargeback report is displayed.

    The chargeback report is generated.

  4. Enter the vCore/Hour or Memory MB/Hour and click Apply. A quick estimation is displayed in the results table, in the CPU hours or Memory hours column of the corresponding group by the table.

In this section: