Friday 26 April 2013

How to pass Context parameters to Talend Job through command line

Today, I am going to show, How to pass the values of the context parameters or variables to the Job once the Job has been exported as executable (batch or shell). In the earlier post (mentioned below) I have shown, How to create context variables and pass values to the context variables with in job. Click on the respective link to know more:

We can export Talend job as stand alone executable in the form of batch or shell scripts. Talend allows us to pass the values of context variables defined in the job from command line. This option/feature comes handy when we have a job where business rule keeps on changing. We can easily parameetrize the parameter whose value can be changed and provide the value of parameter through command line. So How can we pass the values from command line.

There are two ways to pass the values of context variables through command line.

1. Using Context Group - We can create different context group in the Talend Job and set the values of the context variables according to these groups. e.g. we can create a context group for Dev (Development environment) , Prod (Production environment) and provide different values to context variables based on the environment. From Talend Job, we can change the context group from Run view as shown below.



Similarly, Use the following command to change the context group of the Talend Job through command line:

For using context group Prod defined in the Job:

C:\demo\context_demo> context_demo_run.bat --context=Prod

For using context group Dev defined in the Job:

C:\demo\context_demo> context_demo_run.bat --context=Dev

If we do not provide the context group then Job will automatically use the default context group defined in the Job.

For using default context group defined in the Job:

C:\demo\context_demo> context_demo_run.bat

2. Overriding Context Variables - We can also override the values of the individual context variables/parameters through command line. We can pass the argument --context_param followed by a name of the parameter defined in the Job and its value to override any of the parameters in the context. Use the following command to override the value of context parameter or variable.

C:\demo\context_demo> context_demo_run.bat --context_param <param-name>=<param-value>

For example: To override the value of context parameter DB_NAME to localhost

C:\demo\context_demo> context_demo_run.bat --context_param DB_NAME=localhost

I have created a simple Talend Job to demonstrate both methods of passing command line parameters:



This Job which will take the values from context parameters and display in Run console.
tFixedFlowInput will be used to put the context parameter values in the Job flow and tLogRow will help to display the values in Run console.

I have created two context parameters param1_origin and param2_origin. I have set the default value as "Inside Job" for both the parameters.


Now, Lets export and run the job through command line:

# First run it without any parameters overriding

C:\demo\context_demo> context_demo_run.bat


Notice that the default value is being displayed.

# Its time to override first context variable param1_origin

C:\demo\context_demo> context_demo_run.bat --context_param
param1_origin="Command Line"



Notice that value from command line has over ridden the default value.

#Now I will pass value of both the context variables through command line:

C:\demo\context_demo>context_demo_run.bat --context_param param1_origin="Command Line" --context_param param2_origin="Command Line"


Now, Modify the same Talend job and  create context group in the Talend Job to demonstrate How to specify which context group to run through command line.


(Note I have deleted the Default context group and set Dev as default)

Now we have two context group Dev and Prod in the Job. Lets import and run the Job though command line.

# Lets first run it without mentioning any context group

C:\demo\context_demo> context_demo_run.bat



Notice that the value from default context group Dev has been populated.

# Now, specify the Prod context group in the Job and pass it through command line

C:\demo\context_demo> context_demo_run.bat --context=Prod



You can also override the value from context group by mentioning the --context_param in the command.

C:\demo\context_demo> context_demo_run.bat --context=Prod --context_param param1_origin="Command Line"



This is How we pass the values to the context variables defined in the Job through command line. Please let me know, what are the best practices you follow when you define context variables and pass the values through command line. Looking for your comments.

Click on the below link to know more about context variables:

6 comments:

  1. Hi,

    Unable to view the images posted. Please help

    ReplyDelete
  2. I asked her to think for five minutes about key issues that she wanted addressed, and were there any open questions left on the Informational Interview table?guarantor

    ReplyDelete
  3. When you override a context from the command line, does that context variable have to also be defined in a context group?
    I have a context variable in a context group call HOME and it is defaulted to no value. When I pass it from the command line with --context_param HOME="C:\home" it does not get set.
    This context parameter sets the home directory for my context file. Subsequently, since it is not set my context file load never succeeds.
    Please advise. Bottom line is that I need to set a HOME directory initially from which the rest of my Talend job works.

    ReplyDelete
    Replies
    1. FOund the issue -- I was using " " around the parameters within the batch file which caused issues.

      Delete