Data Assets Materialization report
Data Assets materialization report creates delta tables with partitions between data based on the selected output type. Partition files are created in the server that is local or uploaded to the Azure container (remote).
Click Generate Reports > New button.
In the New Report dialog box, enter the following details.
Items
Description
General
Name
Name of the report.
Environment
Select your platform. The reports corresponding to the selected platform are only listed in the Report type. If you select the All option, all the reports are listed.
Report type
Type of report. Select Data Assets Materialization.
Schedule
Select the checkbox to schedule the report to run daily, hourly, weekly, or monthly. You can also set the schedule can using a cron expression. You can expand the Example drop-down and select the corresponding options. The next four sample run times are displayed for reference.
Retention
The number of days to retain the report files. All the reports are stored in the
unity-one/src/assets/reports/jobsdirectory. After completion of the retention period, the report files are automatically purged.Parameters
Look Back
Specify in days the period range that can be used to generate a report.
Use Exact Date-Time
Check this option and select the from and to dates to schedule the report for the exact time range specified. The only anomaly with this option is that the same report will be scheduled to be generated repeatedly.
Data Sets
Select a data set. You can select multiple data sets. Ensure to select similar data sets, that is with the same compute modules, else the report is not generated.
Output Type
Select either Local or Remote. If you select Local, the output is stored in the local server. If you select Remote, the files are uploaded to the Azure container.
Output Path
Provide the path where the output can be stored.
Azure Account Name
Specify the Azure account name and container where the partition files should be uploaded. When you select the Remote option for Output type, then this field is mandatory.
File Blob Path
Specify the path to the Azure container where the partition files should be uploaded.
Notifications
Email to
Email ID to send the notification when the report is generated. You can specify multiple email IDs with comma separation. You can also select the Attach Files to Email checkbox to receive the reports as an attachment.
Advance Options
Profile Memory
Select this option if you want to generate logs that help to troubleshoot scenarios where the report takes excessive time to generate or fails to generate.
Caution
The option will increase the report run time excessively.
Click OK. The generated reports will be listed under Reports on the App UI.
Select the generated report and then click Run. After the report is successfully run, the details of the report runs are listed in the Run box on the right.
Click the following:
HTML files link to view the report details.
Input parameters link to view the parameters you chose to run the report.
Log file link to view the logs of the report.
The partition files are uploaded to the specified path.
