What is tagging?
Tagging is a method to apply more context to your applications so you can view and organize them more specifically based on your requirements and needs.
There are two types of tags:
Application tags provide ways to filter applications.
Workflow tags logically group a set of applications.
Application tagging
By the time your application reaches the cluster, much of its context is lost, for example, the dept which submitted it, what project it belongs to, and so on.
All applications have the following metadata. (See how the metadata fields are populated.)
App type
App Id
App Name
Username
Cluster (empty for server-less applications)
Queue (empty for server-less applications)
Configuration (accessible for some applications)
As an admin, you need to be able to organize/view the applications running on the cluster with more granularity than this limited set allows. You will want to “slice and dice" applications in a myriad of ways for various reasons.
Unravel can not “magically” deduce which applications belong to what tenants, projects, teams, etc. to gain such granularity, you must provide Unravel with this information.
Tagging an app is how you tell Unravel about its context.
What is a tag
At the basic level, a tag is simply a <key, value> pair associated with an application.
A simple way to conceptualize tagging is to think of a spreadsheet. The key is the column header, and the value is an entry in the column. For each application, you enter your value, if any, in the column. The following example has two keys, Tenant and Project, which can be used to filter applications. While an app can have multiple tags, it cannot have multiple values for a given key. In this example, an app cannot be tagged with proj1 and proj2,
App Type | App ID | App Name | User | Cluster | Queue | Tenant | Project |
|---|---|---|---|---|---|---|---|
MR | j_134 | distc | John | def | root.UM-proj1.print | marketing | proj1 |
Spark | s_345 | rate.spark.sim | Jane | def | root.UM-proj2.print | sales | proj2 |
Spark | s_456 | over.spark.sim | Jane | def | root.UM-proj3.print | sales | proj3 |
See the following Example for an explanation of how these tags were generated.
Effective use of tags
You can use tags to:
Generate chargeback reports based upon specific criteria, e.g., project, dept, team, etc.
Decide which apps a specific user can see, e.g., the marketing head can see all marketing apps, while user2 can only see specific marketing project apps.
Group applications together. (See Tagging workflows.)
To effectively use tags, you need to understand your requirements for displaying or grouping your apps. For example:
Do you need chargeback reports for each tenant in a multi-tenant cluster? The apps must be tagged with the tenant it belongs to.
Do applications need to be billed back to departments and teams? Apps need department and team tags.
Do you want some users to see all projects and others just a subset of the projects. Apps must have project tags.
A specific tag can be used for different purposes. You can use the project tag to generate chargeback reports and filter views specific to users.
Assigning tags
You generate a tagging dictionary via a Python script. Unravel then uses the dictionary to apply tags to the applications as you prescribe.
But like Unravel, the only information you have about the application running on the cluster is its metadata. So how can you develop a script to tag specific applications; how can you determine and generate your <key, value> pairs?
Methods to generate/create tags for an application
By creating naming conventions for apps, queues, and cluster names, you can embed information for your tags, e.g., placing all apps belonging to project-1 in the root.UM-proj1.print queue lets you extract the project name from the queue the app is in.
You can use the app’s metadata to create the tag values, for example:
Directly, e.g., <team,username>
Parse it to extract information, e.g., <project, {extracted fromqueue name}>
Concatenate metadata with other metadata or strings, e.g., <dept, {username}+ {extracted fromapp name} + string>
A tagging script can access files that contain further mapping information. e.g., maps projects to tenants.
You can download example tagging scripts. (This is currently private; contact Unravel Support.)
Example
The above table is a simple example to help you understand tagging concepts. In your environment, you will likely use a more complicated schema. In this example, we show how the tags were created.
The cluster is multi-tenant; we created the tags:
<tenant, marketing>
<tenant, sales>
There are three projects; we created the tags:
<project, proj1>
<project, proj2>
<project, proj3>
Finally, we have a file that maps the projects to the tenants:
proj1 to marketing
proj2 and proj3 to sales
We then told Unravel about the tags and how to assign them, that is, developed the tagging dictionary.
First, an app’s queue is parsed to extract the project it belongs to. The project name is encoded between "UM-" and ".". Once extracted, the name was used as the project value. Next, the script accessed a file that mapped the project name (value) to the tenant (key).
The script resulted in three applications being assigned tenant and project tags.
Workflow tagging
Workflow tags are much simpler than application tags. You use preexisting Unravel tags to create workflows, specifically unravel.workflow.name and unravel.workflow.utctimestamp. See Tagging workflows for more information on creating workflows.