Discussions
Categories
Groups
Community Home
Categories
INTERNAL ENABLEMENT
POPULAR
PUBLIC CLOUD
PRIVATE CLOUD
Quick Links
MY LINKS
HELPFUL TIPS
Back to website
Home
Intelligence (Analytics)
Large intermediate file
stevegal
I'm trying to generate a report for my application. Basically it contains a around 6500 records with 15 or so data elements per record (each element is relatively small). When I get the run task to write it's output using a filearchivewriter the size of the resulting file is a massive 8 gig! I'm using birt 2.6 at the moment, but this file is just astoundingly large. Eventually I need to cope with many requests, and that doesn't sound right to me for a resultant report that, in uncompressed HTML is around 1 meg.
Any ideas what I may be doing wrong? Maybe there is a configuration setting I can use to get something more reasonable?
Find more posts tagged with
Comments
stevegal
I've had a quick look at the output when I use the folder archive. I see a folder per row of output, but inside each folder there seems to be a copy of the full dataset. Is this really how it works, or have I am I missing some config somewhere so it can reference one dataset?
JasonW
This does not sound normal. If you generate a rptdocument instead what is the size?
Jason
stevegal
thats much smaller (don't have the exact size off-hand). Been looking into the BIRT source and it seems to write out all the results for each row of data. I've spotted a flag that is named useCache and its always false in each data element. I'm guessing thats why its repeating the dataset everytime for each row. Is that flag settable in the report design? (from my brief search so far, it seems that if I have a datasetref in the dataelement then the use cache flag would be true (is it also set if a set the dataset on each data element?).
Help much appreciated!
stevegal
Hmmm. I take that back. it is smaller, but only because it seems to be broken. Again, looking inside it, it would appear to have the result from the database repeated over and over again. I'm sure this must be something obviously worng with my report design... just need pointing in the correct direction
JasonW
You sure the design is working properly? In the desginer in the menu there is an option to generate a report document. Can you do that and verify the size?
Jason
stevegal
Its around 110Meg. Surely much too large....
stevegal
I've just found a flag on the dataset in the advanced properties ... "needs dataengine cache" or something. With this set to true, the folder structure you get is much better, and doesn;t repeat the same data for each row.
Hope this helps others... I think its got me over my hurdle for now...
JasonW
Thanks for posting your solution
