BIRT Engine integration in pentaho kettle PDI - svg image files badly formatted

Hi, 
 
I am working on a birt output step for pentaho data integrator (pdi,kettle). 
The version of birt framework I work with is 4.2.1. 
The step is working good in pdi(kettle) 4.1, it can output pdf, xls, html files 
As pdi 4.1 starts to be an old version now, I am trying to adapt it to pdi 4.3. 
There come my problems, the charts do not come out using version 4.3. 
Same thing with version 4.4 
After some researches, I found out that only svg image do not work. 
when generating an html file using the step, the svg image that is created differs from the one create with version 4.1: 
- the <pre class='_prettyXprint _lang-auto _linenums:0'><![CDATA[</pre> tag that contains the script part of it does not exist in 4.3 version 
- all the <pre class='_prettyXprint _lang-auto _linenums:0'>&, <, ></pre> of the script are replaced whith <pre class='_prettyXprint _lang-auto _linenums:0'>& < ></pre> that creates an unreadable code 
- end of lines used for 4.3 version are LF instead of CRLF 
 
Adding the <pre class='_prettyXprint _lang-auto _linenums:0'><![CDATA[</pre> tag and correcting the script make the charts appear. 
I believe it comes from either: 
 
- one of the libraries of pdi 4.3 that interfers with the birt ones (but there is such a huge amount of them) 
- some local that could be set somewhere (OS locale is french) 
 
Any idea ?

Find more posts tagged with

Comments

CBR

Hi,

are you author of:
http://b-e-o.blogspot.de/ ?

I know both projects (PDI and BIRT) quite well and i think that it is much likely that some libs are interfering. On the other hand PDI creates its own classloader for each step plugin.
Can you try to use the osgi runtime instead of the default non osgi one? It would be interesting to see if it works then.

In addition i m very interested in what you are doing. Thought about developing something like that myself but then i found your blog and gave up on it.
I would be happy to help if you are interested and your project setup allows collaboration.

gildas

Hi,

I am not.
This step seems to be the one I started from (I thought it was from Matt Casters though).
Wanted to be able to manage dynamic output format, dynamic parameters, and use newer version of Birt, so I did a few changes.
I am interested in project collaboration, I was thinking of opening a community project, but I have no experience in this kind of thing.

Gildas

CBR

Hi Gildas,

let me know if you want me to join :-) If you prefer to develop it on your own it would also not be a problem as i could just join for a few side topics (only thing i want is to be able to use your plugin without paying for it :-))

You are right the existing plugin lacks of some important features. It would be also thinkable to let it create .rptdocuments. A lot of customers have the need to batch generate the rptdocuments during the night. In addition i also have some additional ideas to enhance the plugin.

The easiest thing is to open a project on googlecode because it is free for open source project. It has svn as well as a website that you could use to place the release downloads.

gildas

After a week of holidays, I am back. 
I created a project and posted the first version of the BIRTOuput step. 
<a class='bbc_url' href='https://code.google.com/p/birt-document-generator-for-pentaho-data-integration/'>https://code.google.com/p/birt-document-generator-for-pentaho-data-integration/</a> 
Still much to do, I have to make GIT work now. 
 
I still need help on that subject, so if anyone has an idea why the svg image is not generated correctly using version 4.3 of pdi. I take it. 
 
Gildas

CBR

sorry was on a business travel last weeks.
Awesome! I ll checkout the sources and try to make it work in PDI.

CBR

Hi,

i was able to make it work using the OSGI runtime instead. Most likely SVG is not working in the non-osgi runtime because of some conflicting jar files. I m not sure which jar is causing the conflict but i think that there is even more then a single jar files that is also part of PDI.
I would suggest to only use the osgi runtime because it isolates BIRT from PDI. Unfortunately BIRT introduces a lot of different third party libraries, so if there is no jar conflicting that time it is much likely that there are in future version or other PDI plugins.

Two other side notes:
- The code doesn't close the tasks. Not closing tasks can cause the runtime to deadlock.
- Platform won't be shut down and is started each time the plugin is processing rows.

I can share my code if you are interested. The plugin looks really good i would just change some implementation details.

gildas

Hi,

Could you reproduce the svg charts problem?

I know nothing about osgi. The thing is, with the firt version of the step, there was a birtEngine path defined and this did not work with the latest version of the birt runtime, so I had to remove it.
Is it about running the birt engine in a separate environment? That would be great

I appreciate your help. This is what I came here for. Do you want to join the project? You're welcome.

By the way, I already worked on generating to and from rptdocuments. I thought it would allow me to isolate where the problem occurs. And it did : the .rptdocuments are almost identical,it's in the rendering. I will upload it tonight.

Gildas

CBR

Hi Gildas,

sure i can share my code. I ll just write you a private message including my google code username.

The path that was there before was pointing to the osgi framework so i m assuming that the original developer also used the osgi version. OSGI is the underlying framework of eclipse. It is quite complex but it allows eclipse plugins to run. The big advantage is that these plugins are isolated from the rest of PDI. In PDI there is a single classloader and therefore you can only have a single version of a third party library. Using the OSGI version BIRT runs in a container. Everything inside the container is invisible to PDI. Therefore no side effects.

You are right. The reason for the bad SVG files are only in the rendering and not in the generation. But again, i m not sure that it is the only issue. If you really want to use the non osgi version i would start analyzing all jar files using JarAnalyzer or something like that to find all conflicting jar files.

Christian

CBR

Oh i ve just seen that they removed the private message feature. Would you mind writing me an email? I do not want to share my email because of Spam.
You can find the email link on my profile.