Discussions
Categories
Groups
Community Home
Categories
INTERNAL ENABLEMENT
POPULAR
PUBLIC CLOUD
PRIVATE CLOUD
Quick Links
MY LINKS
HELPFUL TIPS
Back to website
Home
Intelligence (Analytics)
BIRT Memory Issues Handling
sarfraz
Hi All,
I have a report which have a query which fetches around 60000 of rows from database, once I execute this report from BIRT Viewer it consumes a lot of memory and time to generate and render this report on UI.
As we know while creating or generating a report BIRT creates corresponding rptdocument(****.rptdocument) in a temporary directory, in our case the report which should show around 60000 rows of data in report takes more than 18GB of disk space for creating this temporary file i.e. ****.rptdocument.
While going through the BIRT document, only solution I could find is by limiting the rows to be fetched from the database which can be done by mentioning that in web.xml, but in our case this will be limitation, i.e. lets say if our report should return 60000 of rows and we limited the rows for 5000 to reduce memory consumption and the report will render only 5000 of rows which will be not the exact report that is required.
Therefore request you all to suggest/comment if there is any solution to this issue.
Thanks and Regards,
Sarfraz
Find more posts tagged with
Comments
Yaytay
Where is the data coming from?
BIRT has to capture metadata for the data sets - in some (most) cases it can only do this by running the query and in some (again, most by default) cases this results in putting the entire data set in memory.
I think BIRT can do better in this area, but part of the blame lies with JDBC and your JDBC driver may provide you with options that can stop it from loading the entire data set in one go (you still have to wait for the query to be evaluated, but it doesn't need to be all loaded into the heap).
sarfraz
Hi,
Thanks for your reply.
Data is coming from MySQL database.
Can you please explain more how can I control the bulk data while creating and rendering a data report which results on million of records, I am generating report through birt viewer web application.
We don't want to restrict on fetching data from database on JDBC Driver level.
Thanks and Regards,
Sarfraz
Yaytay
Sorry about the delay in replying, I've had to rebuild my computer and it's still not all there
What I found to be the problem is that BIRT runs the query early on in order to get the metadata for the resultset.
The reason this is a problem is that by default the MySQL JDBC driver loads the entire resultset into memory when it the query is execute.
To avoid this killing your process you need to tell the JDBC driver to only load the data bit by bit.
I have found the following JDBC URL enables me to run queries that pull in tens of thousands of rows without exhausting all memory:
jdbc:mysql://192.168.1.198/test?useServerPrepStmts=true&useCursorFetch=true&defaultFetchSize=1000