Is BIRT right for me?

baj09

Hi,

I have heard and seen many good things about BIRT and would like to investigate further if we can use it for our purposes...

We are running a core facility in a research institute and are producing QC data with various programs. There is only a finite (<30) number of such QC files with varying text formats, most of them are human readable, with varying number of lines, sometimes tab-del/CSV, sometimes plain text with numbers at particular positions (In these cases I can identify key words and extract numbers from those lines)...

I would like to collect a set of QC values from these files and compile them in one PDF/HTML document. I would also like to compile results from different experiments, generate graphs with potentially many millions of entries and compile everything...

From the BIRT-examples I know now how to generate a report from a standard (e.g. TSV file) file. I have seen that one can write java applications using the API to control BIRT, but these programs would be too specific for us; we can write parsers for the "human readable" files, but I haven't seen an example for this...

After spending some hours going through the tutorials and googling around, I decided to compile and post my questions here. Please let me know if this is not the right forum and where I should rather post them...

- I haven't found information on how to go about "human readable"/in-house/very specific file formats; where would I start looking?

- Would it be possible to construct something like a template for each file type and then copy/paste those to construct a new report? This would enable a Non-programmer to construct a report by using the Eclipse GUI...

- I want to construct a table from an unknown/varying number of input files that all have the same format (e.g. collect the last line from all these files) would this be possible without specifying a new explicit data source for each of them? Basically I would like to specify a directory and then the input files can be found automatically, or I could use a table with the file names...

- What is the limitation of number or rows/columns that can be processed by BIRT to construct a plot (line plot, with ~32GB of Mem)

- I read about the runtime/POJO, but I couldn't find any example of where/how I can change a file name/data source. In fact I didn't find much documentation on the ReportEngine at all, only the API... Where could I find this?

- Is BIRT the right tool for us? We are also considering R/Sweeve/Latex... What would be the advantage of using BIRT?

I am pretty sure that most/all of these questions are pretty basic, but unfortunately I haven't found an answer to them... Please forgive my ignorance...

Thanks a lot for your time and consideration...

Bernd

johnw

1. For the human readable/in house/very specific file formats, I would consider writing an external ODA. I would consider using a good parser like Antlr for this kind of parsing. The specific grammar to parse the format you describe shouldn't be very difficult to write.

2. Yes. Templates are supported out of the box. Any report design can be promoted to a template. Libraries are also supported for re-use of components.

3. There are a couple different ways. The easiest way I can think of is using a Scripted Data Source using Javas standard File IO. Create a File object on the directory, iterate through the files reading the last line and adding to the data set.

4. Thats a Jason Weathersby question

5. The POJO data source is not provided with the open source BIRT, it is in the commercial BIRT. To use POJO's in a BIRT report using the open source version, the Scripted Data Source is the way to go. The best reference for the Report Engine is "Integrating and Extending BIRT", available from Amazon.

6. BIRT might be the right tool. You're going to know that better than anyone else since you know your requirements better. BIRT is really good at BI type applications. So, collecting, aggregating, calculating, and displaying information from several different sources, data bases, data warehouses, files, web services...

With that said, it sounds like with a little extending, BIRT should work find for you. I couldn't tell you if BIRT will serve you better that Sweave since I haven't worked with Sweave directly. In the end it will boil down to which would you prefer to work with, R or Java. With Latex you will have more fine grained control of your output, but it will require a lot more coding and tweaking than with BIRT. With BIRT, you have the avenue of using either JAva or Javascript (depending on your needs and complexity), can get a good visual approximation of what output will look like in the designer. And its going to be easier for your non-coders to work with.

Hope that helps. About the memory concerns, I don't think you will run into issues, but talk with JWeathersby, he can answer more definitively.