Managing Job Configuration
After you have defined your data sources, which specify the locations of your input and output data, you can then create a job configuration. The job configuration uses these data sources and outlines the specific transformation tasks to be performed on the data.
A job configuration is required to specify details about the data to be transformed and the destination where the transformed data will be stored. The job configuration can be created using the CipherTrust Manager UI or API playground.
Note
In this section, the terms BDT client and client have been used interchangeably.
All objects related to files and databases–such as table names, column names schema names, and user names—are case-sensitive. Their casing must exactly match how they are stored in the database or file.
When a job is run, the BDT client that sends the heartbeat first will run that job using the latest configurations. When the BDT client is going to perform the operation, it sends the In Progress status to the CipherTrust Manager. If the BDT client fails to process the request, the job is Stopped with error.
The status of the job is displayed on the Job Status page of the CipherTrust Manager UI.
Prerequisites
A data source should exist on the CipherTrust Manager.
A protection policy should be created on the CipherTrust Manager.
In this article, you will learn how to:
Creating Job Configuration
To create a job configuration:
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations > Add Job Configuration. The Add Job Configuration Wizard is displayed. Follow the steps to complete the setup.
Add General Info
On the General Info screen, enter the following details:
Field Description Default Mandatory Name Friendly name for job configuration. Yes Description Description for job configuration. No Unprocessed Record File Path The absolute path (without file name) inside the container where the unprocessed records will be captured during transformation. If this path is not specified, the file will not be created. If you specify the location for Unprocessed Record File Path, files with below formats (based on the source type) will be created at the specified location as shown here:
— Database:<job_id>-<source_table>-<timestamp>.txt
— File:<job_id>-<timestamp>.txtIt is recommended to configure this path only if it is required as it contains sensitive information. If configured, you must delete the created file after use or manage its access control in order to prevent the sensitive information from invalid access.
No Click Next to go to the Data Source Info screen.
Data Source Info
The source and destination data-sources can be one of the following:
File to file (File-to-File)
Database to database (DB-to-DB)
Database to file (DB-to-File)
File to database (File-to-DB)
Click the desired tab to view the instructions to configure data sources.
On the Data Source Info screen, select the following details:
Field Description Default Required SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Mandatory DESTINATION data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Mandatory Click Next to go to the Table and Column Info screen.
Table and Column Info
On the Table and Column Info screen:
- Click Add Table/Column.
Note
The source data-source can't be modified.
(Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Required Offset Start record index. 0 Optional Limit Maximum number of records/rows to be transformed, starting from offset. 0 Optional Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 Optional Under the FILTERS section, specify the following details:
Field Description Default Required Selector Name of the column. Mandatory to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = Mandatory to configure filter. Expression Evaluates filter on selector using the specified operator. Example of Filter:
Salary > 10000
Here,Salaryis the Selector,>is the Operator,10000is the Expression.Mandatory to configure filter. Similarly, you can add more filters by clicking the + Add More Filters link.
Click Next to go to the Column Info screen.
On the Column Info screen, enter/select the following details.
Fields Description Default Required Source Column Name Name of the source column.
The source column can't be null, empty; also, it should not start or end with space.Mandatory Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. Optional Target Column Name Name of the target column. Mandatory when Same as Source check box is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectMandatory Source Column Action Action to be performed on the source column in the destination table after transformation.
Supported actions are: PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source and version header columns.PRESERVE Mandatory when Same as Source check box is not selected. Protection Policy Protection policy to be used. If the protection policy doesn't exist, create one. Mandatory Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.Mandatory when external versioned protection policy is used in transformation. Note
Fixed length file in destination is only supported with FPE algorithms.
When the destination is Fixed Length File, BDT does the following changes in the version header field:
If the value of
version header columnlength is greater than 7, spaces are added in the version header value.If the value of
version header columnlength is less than 7, spaces are added in the version header name.If the
version header columnis already present in source, the column length of the original column will be used.
Similarly, you can add more columns by clicking the + Add More Columns link.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Required SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Mandatory Same as Source If selected, the output of the transformation will be written in the Source Column Name field. If not selected, the output of the transformation will be stored in the Target Column Name field. Optional Destination data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Mandatory when Same as Source check box is not selected. Click Next to go to the Table and Column Info screen.
Table and Column Info
On the Table and Column Info screen:
Click Add Table/Column.
Under the SOURCE section, enter the following details:
Field Description Default Required Table Name Source table name to be transformed. Mandatory Schema Owner (Schema) of the source table. Optional (Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Required Offset Start record index. 0 Optional Limit Maximum number of records/rows to be transformed, starting from offset. 0 Optional Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 Optional Order By Sort the selected records. Optional Order Let user select the sorting order of the records. Supported options are
— Ascending
— DescendingAscending Optional Under the FILTERS section, specify the following details:
Field Description Default Required Selector Name of the column. Mandatory to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = Mandatory to configure filter. Expression Evaluates filter on selector using specified operator. Example of Filter:
Salary > 10000
Here,Salaryis the Selector,>is the Operator,10000is the Expression.Mandatory to configure filter. Similarly, you can add more filters by clicking the + Add More Filters link.
Under the DESTINATION section, enter the following details:
Field Description Default Required Table Name Destination table name where the data will be transformed. Mandatory Schema Owner (Schema) of the destination table. Optional Create Destination Table if it does not exist Select to create destination table if it doesn't exist. Optional Click Next to go to the Column Info screen and enter the following details:
Field Description Default Required Source Column Name Name of the source column.
The source column can't be null, empty; also, it should not start or end with space.Mandatory Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. Optional Target Column Name Name of the target column. Mandatory when Same as Source check box is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectMandatory Source Column Action Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source and version header columns.PRESERVE Mandatory when Same as Source check box is not selected. Protection Policy Protection policy to be used. If the protection policy doesn't exist, create one. Mandatory Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.Mandatory when external versioned protection policy is used in transformation. Similarly, you can add more columns by clicking the + Add More Columns link.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Required SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Mandatory Same as Source If selected, the output of the transformation will be written in the Source Column Name field. If not selected, the output of the transformation will be stored in the Target Column Name field. Optional Destination data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Mandatory when Same as Source check box is not selected. - Click Next to go to Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table/Column.
Under the SOURCE section, enter the following details:
Field Description Default Required Table Name Source table name to be transformed. Mandatory Schema Owner (Schema) of the source table. Optional (Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Required Offset Start record index. 0 Optional Limit Maximum number of records/rows to be transformed, starting from offset. 0 Optional Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 Optional Order By Sort the selected records. Optional Order Let user select the sorting order of the records. Supported options are
— Ascending
— DescendingAscending Optional Under the FILTERS section, specify the following details:
Field Description Default Required Selector Name of the column. Mandatory to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = Mandatory to configure filter. Expression Evaluates filter on selector using specified operator. Example of Filter:
Salary > 10000
Here,Salaryis the Selector,>is the Operator,10000is the Expression.Mandatory to configure filter. Similarly, you can add more filters by clicking the + Add More Filters link.
Click Next to go to the Column Info screen and enter the following details:
Field Description Default Required Source Column Name Name of the source column. The source column can't be null, empty; also, it should not start or end with space. Mandatory Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. Optional Target Column Name Name of the target column. Mandatory when Same as Source check box is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectMandatory Source Column Action Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source and version header columns.PRESERVE Mandatory when Same as Source check box is not selected. Protection Policy Protection policy to be used. If the protection policy doesn't exist, create one. Mandatory Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.Mandatory when external versioned protection policy is used in transformation. Note
Fixed length file in destination is only supported with FPE algorithms.
When the destination is Fixed Length File, BDT does the following changes in the version header field:
If the value of
version header columnlength is greater than 7, spaces are added in the version header value.If the value of
version header columnlength is less than 7, spaces are added in the version header name.If the
version header columnis already present in source, the column length of the original column will be used.
Similarly, you can add more columns by clicking the + Add More Columns link.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Required SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Mandatory DESTINATION data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Mandatory Click Next to go to Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table/Column.
The source data-source can't be modified.
(Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Required Offset Start record index. 0 Optional Limit Maximum number of records/rows to be transformed, starting from offset. 0 Optional Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 Optional Under the FILTERS section, specify the following details:
Field Description Default Required Selector Name of the column. Mandatory to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = Mandatory to configure filter. Expression Evaluates filter on selector using the specified operator. Example of Filter:
Salary > 10000
Here,Salaryis the Selector,>is the Operator,10000is the Expression.Mandatory to configure filter. Similarly, you can add more filters by clicking the + Add More Filters link.
Under the DESTINATION section, enter the following details:
Field Description Default Required Table Name Destination table name where the data will be transformed. Mandatory Schema Owner (Schema) of the destination table. Optional Create Destination Table if it does not exist Select to create destination table if it doesn't exist. Optional Click Next to go to the Column Info screen and enter the following details:
Field Description Default Required Source Column Name Name of the source column. The source column can't be null, empty; also, it should not start or end with space. Mandatory Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. Optional Target Column Name Name of the target column. Mandatory when Same as Source check-box is not selected. Source Column Action Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source and version header columns.PRESERVE Mandatory when Same as Source check box is not selected. Protection Policy Protection policy to be used. If the protection policy doesn't exist, create one. Mandatory Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.Mandatory when external versioned protection policy is used in transformation. Similarly, you can add more columns by clicking the + Add More Columns link.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
Confirmation
On the Confirmation screen, verify the job configuration details. The Confirmation screen displays general information, data source details, and table and column information.
If you want to modify any detail, click Edit and update the details.
Click Add Job Configuration.
Click Close to exit the setup.
Viewing/Editing Job Configuration
Viewing details of job configuration
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The Job Configurations page displays the following details:
| Column | Description |
|---|---|
| Name | Name of the job configuration. |
| Source Data Source | Name of the source data-source. |
| Destination Data Source | Name of the destination data-source. |
| Created On | Date and time when the job configuration was created. |
| Last Modified | Time when the job configuration was last modified. |
| Description | Description of the job configuration. |
Editing details of job configuration
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The list of job configuration is displayed.
Click the overflow icon (
) corresponding to the job configuration that you want to edit.Tip
You can also click the job configuration name to edit the details.
Click View/Edit. The mini detail view shows the details of job configuration, such as general info, data source, table and column info. On this page, you can modify the following fields:
Description of the job configuration
Path of unprocessed record file
Source and destination data sources
Note
The data source type can't be switched between database and files.
The CSV data source type can't be changed to Fixed Length File and vice-versa.
Table and column details
You can either modify the existing tables/columns or add more tables/columns.
To modify existing table
Click the overflow icon (
) corresponding to the table name. You can edit the following fields:Source Column Name
Target Column Name (only editable If the Same as Source check box is not selected)
To modify existing columns
Click the Right arrow (
) to display the list of configured columns.Click the overflow icon (
) corresponding to the column name to be modified and click Edit. You can edit the following fields:Source Column Name
Same as Source check box
Target Column Name (Only editable when the Same as Source check box is not selected)
Operation
Source Column Action (Only editable when the Same as Source check box is not selected)
Protection Policy
To add new tables and columns, click Add Table/Column or Add Column respectively.
Save your changes.
Click Update.
Deleting Job Configuration
To delete a job configuration:
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The list of job configurations is displayed.
Click the overflow icon (
) corresponding to the job configuration that you want to delete.Click Delete. A message appears prompting to confirm the job configuration deletion.
Deleting a job configuration is permanent and cannot be undone.
Click Delete. The job configuration is deleted.
Next Topic
After the job configuration is created, you can run the job from the:
