Sunday 20 January 2013

Understand Context variables with Examples - Part 2 (Talend Open Studio)


In the previous post here, I have shown you, How to define and use context variable in the Talend Job. Today, I am going to define context variables in the Repository as metadata. You can also call these variables as Repository context variables. We define context variable in the repository if the context variables needs to be available to multiple jobs in the project. You can access the values of these variables in multiple Talend Jobs.

Today I am going to create a Talend job which will filter the records based on the repository context variable values. Lets look at the input file.



I have formatted the input file as below to make it easy to read.


I will create two context variables; context_dept_id and context_salary.

Lets get started and create a new talend job and follow the below mentioned steps:

1. Create metadata for our Input delimited employee file. You can take reference of the steps to create metadata for delimited from here. Your metadata should look like following:


2.Drag the metadata created in step 1 to Job designer pane and select tFileInputDelimited component from the Popup.

3. Drag tFilterRows and tLogRow components from Palette pane. Component tFilterRows  will be used to filter the records based on the values of the context variables and tLogRow will be used to display the output in output console.

4. Right click tFilterRows, select Row > main and connect it to tLogRow.

As of now, we have not defined any context variable. Lets run the job and observe our output.


Output without context variable: 

99designs.com



As you can see that, all of the input file records have been displayed in the output as we have not defined and used context variable in the tFilterRows component.

Now lets define context variables.

5. Right click Contexts tab in Repository pane and click on Create context group.


6. In the Popup window, enter the name of the context variable group name. Click Next
.
7. In step 2, Click on (+) button to add two context variables context_dept_id and context_salary.


8. Navigate to “Value as tree” tab and select prompt checkbox for both the variables. This will allow us the set the value of the context variable from the prompt at runtime.


9. Now we have successfully defined our context variables in the repository. However these variables are not present in our job. You can check this by opening the Contexts pane of the Job.

10. To make available these context variables to our job, Drag the repository context group to our job. Now again open the Contexts pane and you can see the variables available to our job.



11. Since these context variables are available to our job, its time to use them to filter the input records. Open the component properties of the tFilterRows and add two conditions as shown below. In the value section provide the name of the context variables.

This completes our job. its time to run and check the output.


12. Run the Job and You will be presented with a prompt window where we can provide the value for both the context variables.


Enter “10” for context_dept_id and “50,000” for context_salary.

Click OK.

Output



You can see that only those records are present in the output where department is 10 and salary is greater than 50000. You can compare it with input file. You can find the snapshot of US input file below:

13. Run this job again and Enter “12” for context_dept_id and “10,000” for context_salary.



You can see that only those records are present in the output where department is 12 and salary is greater than 10000. You can compare it with input file (screenshot below).

 

This is How, we define and use context variables in the Repository as metadata. We can use these context variables in multiple talend job. You have to drag the repository context variables in the Job to make it available. In the next article, I will show you, How to define context variable in Flat files which can be loaded in the job at runtime using tContextLoad component.

You may also like to read..

1. Understand Context Variables Part 1 ( Define context variables inside Talend job)
2. Understand Context Variables Part 3 (Populate the values of context variables from file. tContextLoad)
3. How to Pass Context Variables to Child Jobs.
4. How to Pass context Variables/ Parameters through  command line.

4 comments: