Thursday 11 July 2013

Fetch Last record from file/flow in Talend Job

I have received lot of queries from Talend users as to How to fetch last record from file or flow. That’s Why I have decided to put up a article on the same.


Lets look at the Job design below. Basically we are generating a record number in the tJavaRow component using Numeric.Sequence() function. Once we have generate record number for every record then sort it descending using tSortRow on the record number field. This will make the last record to appear first and then we can use tSampleRow component to select only First record by setting the Range as “1”.


This is code for generating record_number in tJavaRow.(Generate Record Number)



output_row.line = input_row.line;
output_row.record_number = Numeric.sequence("recNo", 1, 1);

Update:

+shrikant patil has shared a better solution to fetch the last record from file/flow. Look at the Job Design below:










Using tFileRowCount component we can find the row count of the file and then set the following value of COUNT global variable of component tFileRowCount to the header part of the tFileInputDelimited component.

((Integer)globalMap.get("tFileRowCount_1_COUNT"))-1

Let me know, if you face any issues.


This article is written by +Vikram Takkar  and published on www.vikramtakkar.com, please let me know, if you see this article on any other website/blog.

No comments:

Post a Comment