Performance issue with XML data source as input stream during report generation

Hi,

i am facing performance issue with BIRT report generation. i have given input as InputStream object and this stream is referred to XML content which is nested in DB as CLOB object.

For report generation, it is taking around 5 minutes of time to process where i have 5000 records information in XML content. However i may need to process 50k records of XML content which will go beyond the accepted time.

Can some one tell me how to achieve it. Is there any alternative way to provide input data smoothly without causing any performance implications.

Find more posts tagged with

Comments

mwilliams

Can you post any java code you use to create/run your report and your design? Also, if you could post a sample of your XML, that might be helpful, too. Thanks!

Hemanth012

<blockquote class='ipsBlockquote' data-author="'mwilliams'" data-cid="116119" data-time="1366306485" data-date="18 April 2013 - 10:34 AM">
Can you post any java code you use to create/run your report and your design? Also, if you could post a sample of your XML, that might be helpful, too. Thanks! </blockquote>
Hi, 
 
Here i attached my java source to create the report.

Hemanth012

<blockquote class='ipsBlockquote' data-author="'Hemanth012'" data-cid="116347" data-time="1367241057" data-date="29 April 2013 - 06:10 AM">
Hi, 
 
Here i attached my java source to create the report. </blockquote>
To avoid the performance issue with XML data source, i have altered my process to use java object with the scripted data source and is able to process even with a large volume of data. Since i just have tested with one table, i am not sure how i can design my template with combination of multiple tables with scripted data source. As of now, following one required clarification. 
 
1. I have written code to fetch my java object in open tab of one data set. If i have multiple datasets which needs same java object, do i need to write the same code (written in my first dataset open tab) in this datset open tab also?

mwilliams

If you create multiple dataSets from the same source, I think you can do the connection to your object in the open of the dataSource. Then, the dataSets can all use that source. Let me know.

Hemanth012

Yes williams, 
 
I am able to keep the common logic across all the datasets in my datasource open tab and able to use that source from datasets. Howerver, am facing an another issue with nested if conditions in fetch tab. 
I have written following code in open tab: 
 
size1=0; 
size2=0; 
size3=0; 
And in fecth tab: 
<pre class='_prettyXprint _lang-auto _linenums:0'>[i]if(size1<pol.size()){ // pol block
ins = pol.get(size1);
if(size2<ins.size()){ // ins block
ct = ins.get(size2);
if(size3<ct.size()){ // ct block
acce = ct.get(size3);
row["one"] = acce.firstVal();
row["two"] = acce.secondVal();
size3++;
return true;
} // ct block
size2++;
return true;
} // ins block
size1++;
return true;
} // pol block
return false;[/i]</pre>
 
When i run the report by keeping some println statements, i found that control is going back to pol block when it finds retruning true from ct block. I am expecting that the control should be inside ct block until the condition 'size3<ct.size()' satisfies but it is not happening. Please suggest me what i am doing mistake here.

mwilliams

The fetch script will start from the top for each row, so that's probably why you're seeing what you're seeing. If you need to flatten out your data, you might do so in the open method and store it in arrays to use in the fetch method to populate your rows. Let me know if I'm misunderstanding.

Hemanth012

<blockquote class='ipsBlockquote' data-author="'mwilliams'" data-cid="116760" data-time="1368556042" data-date="14 May 2013 - 11:27 AM">
The fetch script will start from the top for each row, so that's probably why you're seeing what you're seeing. If you need to flatten out your data, you might do so in the open method and store it in arrays to use in the fetch method to populate your rows. Let me know if I'm misunderstanding. </blockquote>
 
I can alter the logic as you mentioned. However, i will not know the count of array objects that i will receive. 
 
I should to iterate through all three blocks to receive all the records. As per the suggetion, i can have the 'pol' array in open tab, but still the remaining two blocks (ins, ct) to be retained in fetch tab only. 
This may not work out?? 
 
I think, there could be some alternative to achieve this either with java code or by using other tabs that are available with scripted data source.

mwilliams

Are you talking about the open tab of the datasource or the dataset? Why can you not process the data in the open method of the dataSet, store it in arrays, then access these arrays in your fetch to assign the values to rows? Maybe I'm misunderstanding. If you could set up a simple example showing me your setup that I can run, maybe that'd help.

Hemanth012

<blockquote class='ipsBlockquote' data-author="'mwilliams'" data-cid="116811" data-time="1368637635" data-date="15 May 2013 - 10:07 AM">
Are you talking about the open tab of the datasource or the dataset? Why can you not process the data in the open method of the dataSet, store it in arrays, then access these arrays in your fetch to assign the values to rows? Maybe I'm misunderstanding. If you could set up a simple example showing me your setup that I can run, maybe that'd help. </blockquote>
Hi Williams, 
 
Yes, by storing all my records into arrays in open tab of dataset resolves my issue. 
Am able to design my template by fulfilling all my requirements. Now i can able to generate a pdf with one lakh of records. 
 
Thanks for your support. 
Cheers...!

mwilliams

Great! Glad I could help! Let us know whenever you have questions!