BIRT Native Excel Emitter Performance Issue

I have been using Birt2.6.2 since long time. Recently I started generating excel reports with huge data(say 13lack rows). Here problem is reports are hanging in Birt and after few hours, failing with gc overload limit exceeded error. It looks like emitter is creating n number of data instances and they are not eligible for garbage collection until or unless count of objects hits max rows limit per page(60k if it is office version 2003). My max heap space limit is 10G. Kindly help me out.

Find more posts tagged with

Comments

jeffmor

Hello,

In case native XLSX format is okay then you are welcome to try our Arctorus BIRT Emitters. It uses our proprietary library to create Office 2007-2010 documents instead of POI and optimized by memory use and speed.

Regards, Jeff

johnw

Any of the native Excel emitters that use POI are going to do that. This includes the Native BIRT Excel emitters that are in dev share and in the Marketplace and the Tribix Excel emitter. The reason is that POI is an incredibly bad memory hog and wasn't really designed for spreadsheets the size that we are talking about.

You have a few options that get around this, and all three cost money. Arctorus is fairly efficient, and is reasonably priced. Then you have your commercial BIRT emitter that comes with BIRT Designer Pro and iServer. This uses a proprietary technology that was designed for spreadsheet reporting, and work extremely well.

Check out the trial versions of both Arctorus AND BIRT Designer Pro. That should give you an idea which will work better for you.

Raghava

Hi Johnw,

Thank you for your quick reply. we are using poi 3.8 beta 4 and it has new package called 'streaming' to address heap memory consumption. so we are not facing any problem from POI side. we profiled Birt prototype excel emitter code found out that more number of data objects causing out of memory issue.It looks like emitter is creating n number of data instances and they are not eligible for garbage collection until or unless count of objects hits max rows limit per page(it is 60k usually).We Integrated Birt with apache POI
to generate 2007 excel file. So in our case,max row limit is 1048576.My max heap space limit is 10G.If I am not wrong,It is wise to flush the data instead of holding it in memory as we have nothing to do with data(sorting,ordering etc).It will help us to cut down heap memory usage. we enabled pagination once,Interestingly report got finished very quickly.I have already raised one bug regarding this

https://bugs.eclipse.org/bugs/show_bug.cgi?id=363673