Managing Job Configuration
After you have defined your data sources, which specify the locations of your input and output data, you can then create a job configuration. The job configuration uses these data sources and outlines the specific transformation tasks to be performed on the data.
A job configuration is required to specify details about the data to be transformed and the destination where the transformed data will be stored. The job configuration can be created on the CipherTrust Manager UI or API playground.
Note
In this section, the terms BDT client and client have been used interchangeably.
All the database objects and column names (for files) are case sensitive.
When a job is run, the BDT client that sends the heartbeat first will run that job using the latest configurations. When the BDT client is going to perform the operation, it sends the In Progress status to the CipherTrust Manager. If the BDT client fails to process the request, the job is Stopped with error.
The state of the job is displayed on the Job Status page of the CipherTrust Manager UI.
Prerequisites
A data source should exist on the CipherTrust Manager.
A protection policy should be created on the CipherTrust Manager.
In this article, you will learn how to:
Creating Job Configuration
To create a job configuration:
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations > Add Job Configuration. The Add Job Configuration Wizard is displayed. Follow the steps to complete the setup.
Add General Info
On the General Info screen, enter the following details:
Field Description Default Mandatory Name Friendly name for job configuration. Yes Description Description for job configuration. No Unprocessed Record File Path The absolute path (without file name) where the unprocessed records will be captured during transformation. If this path is not specified, the file will not be created. If you specify the location for Unprocessed Record File Path, files with below formats (based on the source type) will be created at the specified location as shown here:
— Database:<job_id>-<source_table>-<timestamp>.txt
— File:<job_id>-<timestamp>.txt
It is recommended to configure this path only if it is required as it contains sensitive information. If configured, you must delete the created file after use or manage its access control in order to prevent the sensitive information from invalid access.
No Click Next to go to the Data Source Info screen.
Data Source Info
The source and destination data-sources can be one of the following:
File to file (File-to-File)
Database to database (DB-to-DB)
Database to file (DB-to-File)
File to database (File-to-DB)
Click the desired tab to view the instructions to configure data sources.
On the Data Source Info screen, select the following details:
Field Description Default Mandatory SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Yes DESTINATION data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Yes Click Next to go to the Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table.
(Optional.) Under SOURCE, in the SUBSET section, specify the following details:
Field Description Default Mandatory Offset Start record index. 0 No Limit Maximum number of records/rows to be transformed, starting from offset. 0 No Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 No (Optional.) Under SOURCE, in the FILTERS section, specify the following details:
Field Description Default Mandatory Selector Name of the column. No. Mandatory only to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = No. Mandatory only to configure filter. Expression Evaluates filter on selector using the specified operator. Example of Filter:
Salary > 10000
Here,Salary
is the Selector,>
is the Operator,10000
is the Expression.No. Mandatory only to configure filter. Similarly, you can add more filters.
Click Next to go to the Column Info screen.
On the Column Info screen, enter/select the following details.
Fileds Description Default Mandatory Source Column Name Name of the source column.
The source column can't be null, empty; also, it should not start or end with space.Yes Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. No Target Column Name Name of the target column. No. Mandatory only when the Same as Source checkbox is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectYes Source Column Action Action to be performed on the source column in the destination table after transformation.
Supported actions are: PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source column and version header column.PRESERVE No. Mandatory only when the Same as Source checkbox is not selected. Protection Policy Protection policy to be used.If the protection policy doesn't exist, create one. Yes Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.No. Mandatory only when external versioned protection policy is used in transformation. Note
Fixed length file in destination is only supported with FPE algorithms.
When the destination is Fixed Length File, BDT does the following changes in the version header field:
If the value of
version header column
length is greater than 7, spaces are added in the version header value.If the value of
version header column
length is less than 7, spaces are added in the version header name.If the
version header column
is already present in source, the column length of the original column will be used.
Similarly, you can add more columns.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Mandatory SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Yes Same as Source If selected, the transformation will happen in the source table of the source data-source. No Destination data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. No. Mandatory only when the Same as Source checkbox is not selected. Click Next to go to Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table.
Under the SOURCE section, enter the following details:
Field Description Default Mandatory Table Source table name to be transformed. Yes Schema Owner (Schema) of the source table. No (Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Mandatory Offset Start record index. 0 No Limit Maximum number of records/rows to be transformed, starting from offset. 0 No Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 No Order By Sort the selected records. No Order Let user select the sorting order of the recordes. Supported options are
— Ascending
— DescendingAscending No Under the FILTERS section, specify the following details:
Field Description Default Mandatory Selector Name of the column. No. Mandatory only to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = No. Mandatory only to configure filter. Expression Evaluates filter on selector using specified operator. Example of Filter:
Salary > 10000
Here,Salary
is the Selector,>
is the Operator,10000
is the Expression.No. Mandatory only to configure filter. Similarly, you can add more filters.
Under the DESTINATION section, enter the following details:
Field Description Default Mandatory Table Destination table name where the data will be transformed. Yes Schema Owner (Schema) of the destination table. No Create Destination Table if it does not exist Select to create destination table if it doesn't exist. No Click Next to go to the Column Info screen and enter the following details:
Field Description Default Mandatory Source Column Name Name of the source column.
The source column can't be null, empty; also, it should not start or end with space.Yes Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. No Target Column Name Name of the target column. No. Mandatory only when the Same as Source checkbox is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectYes ACTIONS Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source column and version header column.PRESERVE No. Mandatory only when the Same as Source checkbox is not selected. Protection Policy Protection policy to be used.If the protection policy doesn't exist, create one. Yes Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.No. Mandatory only when external versioned protection policy is used in transformation. Similarly, you can add more columns.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Mandatory SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Yes Same as Source If selected, the transformation will happen in the source table of the source data-source. No Destination data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. No. Mandatory only when the Same as Source checkbox is not selected. - Click Next to go to Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table.
Under the SOURCE section, enter the following details:
Field Description Default Mandatory Table Source table name to be transformed. Yes Schema Owner (Schema) of the source table. No (Optional.) Click Show Subset/Filter Options. This action allows you to configure subsets and filters.
Under the SUBSET section, specify the following details:
Field Description Default Mandatory Offset Start record index. 0 No Limit Maximum number of records/rows to be transformed, starting from offset. 0 No Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 No Order By Sort the selected records. No Order Let user select the sorting order of the recordes. Supported options are
— Ascending
— DescendingAscending No Under the FILTERS section, specify the following details:
Field Description Default Mandatory Selector Name of the column. No. Mandatory only to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = No. Mandatory only to configure filter. Expression Evaluates filter on selector using specified operator. Example of Filter:
Salary > 10000
Here,Salary
is the Selector,>
is the Operator,10000
is the Expression.No. Mandatory only to configure filter. Similarly, you can add more filters.
Click Next to go to the Column Info screen and enter the following details:
Field Description Default Mandatory Source Column Name Name of the source column. The source column can't be null, empty; also, it should not start or end with space. Yes Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. No Target Column Name Name of the target column. No. Mandatory only when the Same as Source checkbox is not selected. Operation Operation to be performed on the column.
Supported options are:
— Protect
— Reveal
— ReprotectYes ACTIONS Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source column and version header column.PRESERVE No. Mandatory only when the Same as Source checkbox is not selected. Protection Policy Protection policy to be used.If the protection policy doesn't exist, create one. Yes Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.No. Mandatory only when external versioned protection policy is used in transformation. Note
Fixed length file in destination is only supported with FPE algorithms.
When the destination is Fixed Length File, BDT does the following changes in the version header field:
If the value of
version header column
length is greater than 7, spaces are added in the version header value.If the value of
version header column
length is less than 7, spaces are added in the version header name.If the
version header column
is already present in source, the column length of the original column will be used.
Similarly, you can add more columns.
Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
On the Data Source Info screen, select the following details:
Field Description Default Mandatory SOURCE data-source Source of the data to be transformed. If the data-source doesn't exist, create one. Yes DESTINATION data-source Destination where the transformed data will be stored. If the data-source doesn't exist, create one. Yes Click Next to go to Table and Column Info.
Table and Column Info
On the Table and Column Info screen:
Click Add Table.
(Optional.) Under SOURCE, in the SUBSET section, specify the following details:
Field Description Default Mandatory Offset Start record index. 0 No Limit Maximum number of records/rows to be transformed, starting from offset. 0 No Recurrency Whether to transform every row in a sequence or skip rows in-between. 1 No (Optional.) Under SOURCE, in the FILTERS section, specify the following details:
Field Description Default Mandatory Selector Name of the column. No. Mandatory only to configure filter. Operator Operator to be used. Some examples are <, >, <= >=, !=, = No. Mandatory only to configure filter. Expression Evaluates filter on selector using the specified operator. Example of Filter:
Salary > 10000
Here,Salary
is the Selector,>
is the Operator,10000
is the Expression.No. Mandatory only to configure filter. Similarly, you can add more filters.
Under the DESTINATION section, enter the following details:
Field Description Default Mandatory Table Destination table name where the data will be transformed. Yes Schema Owner (Schema) of the destination table. No Create Destination Table if it does not exist Select to create destination table if it doesn't exist. No Click Next to go to the Column Info screen and enter the following details:
Field Description Default Mandatory Source Column Name Name of the source column. The source column can't be null, empty; also, it should not start or end with space. Yes Same as Source If selected, the details of the SOURCE data-source will be replicated in the DESTINATION data-source. No Target Column Name Name of the target column. No. Mandatory only when the Same as Source checkbox is not selected. ACTIONS Action to be performed on the source column in the destination table after transformation.
Supported actions are:
— PRESERVE
— EMPTY
— REMOVE
— This field is only configurable when the target column is specified.
— If operation is protect, this field is applicable to source column.
— If operation is reveal, this field is applicable to source column and version header column.PRESERVE No. Mandatory only when the Same as Source checkbox is not selected. Protection Policy Protection policy to be used.If the protection policy doesn't exist, create one. Yes Version Header Column Name of the column that will store the version header details.
The version header column can't be null, empty; also, it should not start or end with space.No. Mandatory only when external versioned protection policy is used in transformation. Click Add. To add more tables, repeat steps mentioned in the Table and Column Info section.
Click Next to go to the Confirmation screen.
Confirmation
On the Confirmation screen, verify the job configuration details. The Confirmation screen displays general information, data source details, and table and column information.
If you want to modify any detail, click Edit and update the details.
Click Add Job Configuration.
Click Close to exit the setup.
Viewing/Editing Job Configuration
Viewing details of job configuration
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The Job Configurations page displays the following details:
Column | Description |
---|---|
Name | Name of the job configuration. |
Source Data Source | Name of the source data-source. |
Destination Data Source | Name of the destination data-source. |
Created On | Date and time when the job configuration was created. |
Last Modified | Time when the job configuration was last modified. |
Description | Description of the job configuration. |
Editing details of job configuration
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The list of job configuration is displayed.
Click the overflow icon (
) corresponding to the job configuration that you want to edit.
Click View/Edit. The mini detail view shows the details of job configuration, such as general info, data source, table and column info. On this page, you can modify the following fields:
Description of the job configuration
Path of unprocessed record file
Source and destination data sources
Note
The data source type can't be switched between database and files.
The CSV data source type can't be changed to Fixed Length File and vice-versa.
Table details. You can either edit the existing table or create a new table. When modifying the existing table, following parameters are editable:
Source Column Name
Target Column Name (only editable if the Same as Source checkbox is not selected)
Click Update.
Deleting Job Configuration
To delete a job configuration:
Log on to the CipherTrust Manager GUI as administrator.
Open Application Data Protection.
In the left pane, click Job Configurations. The list of job configurations is displayed.
Click the overflow icon (
) corresponding to the job configuration that you want to delete.
Click Delete. A message appears prompting to confirm the job configuration deletion.
Deleting a job configuration is permanent and cannot be undone.
Click Delete. The job configuration is deleted.
Next Topic
After the job configuration is created, you can run the job from the: