scripted dataset to parse a dynamic xml string

Hi,
 
I am working on a report for a customer and the customer is using webservice that returns a dynamic xml string. The xml string has all the data and it changes based on the service request and paramters.
 
Suppose that the attached test3.xml is returned as a string in dataset1. Now I want to write two different scripted datasets to parse the output xml string of dataset1 and creates the desired outputs in birt datasets.xls, can someone please help me write the scripted datasets to parse the xml and get the desired outpts
 
test3.xml is the string that I get from webservice
Birt Datasets.xls two worksheets is the output that I want throught he scripted datasets
 
Many thanks
Arif

Find more posts tagged with

Comments

Clement Wong

There is a very detailed blog post about transposing data using a scripted data source. You can read about the technique here:
 
<a data-ipb='nomediaparse' href='https://enterprisesmartapps.wordpress.com/2013/02/18/columns-to-rows-in-birt-data-set/'>Part 1</a>
<a data-ipb='nomediaparse' href='https://enterprisesmartapps.wordpress.com/2013/02/22/birt-columns-to-rows-in-data-set-part-2/'>Part 2</a>
 
Also, the attached sample written by a colleague uses a scripted data source. See the "Transposed" Data Set.

Arif shah

Hi Many thanks. My first challenge is to parse the xml string and convert them into some shape in rows and columns. My webservice is returning a string that has a big xml data and I dont know how to parse that into rows and columns. Please have a look at the xml file that I have attached. This is what is being returned as a string from the webservice
 
I will appreciate if yoiu could guide me how to parse this kind of xml into rows and columns dynamically
 
Arif

Clement Wong

Is the XML in the test3.xml file, the actual text returned from your web service?
 
If so, it is not valid XML. To double check, I ran the XML through two different XML Validators (<a data-ipb='nomediaparse' href='http://www.w3schools.com/xml/xml_validator.asp'>here </a>and <a data-ipb='nomediaparse' href='http://www.xmlvalidation.com/'>there</a>), and there are errors.
 
Can you please fix?

Arif shah

I apologize for attaching an invalid xml. Here is the valid test xml file
 
Syed

Clement Wong

To consume your specific XML data source with a custom format, your best option is to write a scripted data source and use your favorite Java SAX or DOM XML parser to read the data and transpose/pivot it.
 
There are other options, but they are not optional or recommended. For illustration purposes only and not for production use, it's possible to have the data in the format before transposing/pivoting by:
 a. Use the existing XML Data Source
 b. Create segmented Data Sets for the columnsName and the linesName data first
 c. Store the values in separate Java ArrayLists
 d. Create a final Items Data Set with computed columns that use the columnName and linesName
 e. Use the two Data Sets created in Step b, but change the Visibility to Hide so that the Data Set is executed to populate the arraylist

Arif shah

Many thanks. I think I am gradually grasping the idea now. Now using your suggestion, I have written some code which can actuall fetch me a single node from the xml string.
<pre class="_prettyXprint">
importPackage(Packages.org.apache.xerces.parsers);
importPackage(Packages.org.xml.sax);
importPackage(Packages.java.io);

var parser = new DOMParser();
parser.parse(new InputSource(new StringReader(dataSetRow["return"])));
var nodelist = parser.getDocument().getElementsByTagName('Item');
var entry = nodelist.item(0).getFirstChild().getNodeValue();

entry;</pre>
Now I have one more question here. The above code only gives me data for a single node. How can I iterate the whole string to fetch data for all nodes?
 
 
Once again, Many thanks for the kind help. I really appreciate the guidance here
 
Arif

Clement Wong

It looks like the same question/code you had another thread @ <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/topic/36265-create-soapuiwsdlwebservices-data-sources-and-dataset-in-birt/#entry137679'>http://developer.actuate.com/community/forum/index.php?/topic/36265-create-soapuiwsdlwebservices-data-sources-and-dataset-in-birt/#entry137679</a> and Jesse provided a few solutions for you.
 
The code above appears to be for a computed column in an existing data set for a single row. Your best bet is to use a scripted data source to iterate over the nodes in the Fetch event. For more info about scripted data source and XML, read a little more @ <a data-ipb='nomediaparse' href='https://wiki.eclipse.org/BIRT/FAQ/Data_Access#Q:_How_do_I_use_XML_as_a_POJO_Scripted_Data_Source.3F'>https://wiki.eclipse.org/BIRT/FAQ/Data_Access#Q:_How_do_I_use_XML_as_a_POJO_Scripted_Data_Source.3F</a>.
 
Please review the sample code Jesse wrote in this other thread where he is using a scripted data source against an XML data source instead of the out of the box XML ODA @ <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/topic/36259-issues-with-xml-data-source/?p=134436'>http://developer.actuate.com/community/forum/index.php?/topic/36259-issues-with-xml-data-source/?p=134436</a>

Clement Wong

As a follow up, I have just posted an example of reading XML using the included <a data-ipb='nomediaparse' href='https://developer.mozilla.org/en-US/docs/Archive/Web/E4X/Processing_XML_with_E4X'>Mozilla Rhino JavaScript extension E4X</a>. It's very simple and easy with E4X. The DevShare example that reads a RSS feed is located here @ <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/files/file/1122-parsing-xml-in-birt-with-e4x/'>http://developer.actuate.com/community/forum/index.php?/files/file/1122-parsing-xml-in-birt-with-e4x/</a>

Arif shah

Hi,
 
I have tried to do it using e4x example that you had sent to me. In beforeopen, I have the following code
<pre class="_prettyXprint">
importPackage(Packages.org.apache.xerces.parsers);
importPackage(Packages.org.xml.sax);
importPackage(Packages.java.io);
importPackage(Packages.java.net);

var result = reportContext.getPersistentGlobalVariable("test");
result = result.replace(/<\?[^>]*>/g,"");

rss = new XML(result);
//totalItems = rss.channel.item.length();

totalItems = rss.response.items.items.item.item.length();</pre>
In fetch event, I have the below code
<pre class="_prettyXprint">
if( currentrow >= totalItems ){
return false;
}

row["typeData"] = rss.items.item[currentrow].typeData.toString();
row["value"] = rss.items.item[currentrow].value.toString();

currentrow += 1;
return true;</pre>
My xml is in the same as I had attached before. I am first getting my xml string from a pgv and then trying to parse it the way in your example. Can you please tell me what mistake am I making here to parse the xml?

Clement Wong

Not sure why you have those importPackage lines in your beforeOpen. Also, not sure why you have the result in PGV.
 
Here's pseudo code of how it might be done:
 
beforeOpen (scripted data set)
1. Open web service to get XML (or open file test3.XML) -- your original requirement was that your customer had a web service and I'm not sure if that has changed.
 
2. Save entire XML into local string (result)
 
3. Remove the first line <?xml version="1.0" encoding="utf-8"?> to make it well formed XML
result = result.replace(/<\?[^>]*>/g,"");
 
4. Use E4X to transform into XML literal
rss = new XML(result); 

5. Now, you will need to loop through each of the columnsName elements and save them into a Java ArrayList.
 
columnsNameArray = new java.util.ArrayList();
var columnsNameLength = rss.items.columnsName.columnsName.length();
 
for (i=0;i<columnsNameLength;i++) {
 columnsNameArray.add( rss.items.columnsName.columnsName );
}
 
6. Also, look through each linesName elements and save them into a Java ArrayList.
 
linesNameArray = new java.util.ArrayList();
var linesNameLength = rss.items.linesName.linesName.length();
 
for (i=0;i<linesNameLength;i++) {
 linesNameArray.add( rss.items.linesName.linesName );
}
 
7. Save the length of items.items.item.item length (for test3.xml, it should return 13)
 
innerItemsLength = items.items.item.item.length();
 
8. Save the length of items.items.item.length (for test3.xml, it should return 14)
 
outerItemsLength = items.items.item.length();
 

The total # of 'rows' you will fetch will be innerItemsLength X outerItemsLength (13 x 14 = 182).
 
 
open (scripted data set)
9. Initialize counters
 
currentrow = 0;
innerItemsCtr = 0;
outerItemsCtr = 0;
totalItems = innerItemsLength * outerItemsLength;
 
 
fetch (scripted data set)
 
10. Check to see if the limit of innerItemsLength * outerItemsLength has been reached.
 
if( currentrow >= totalItems ){ return false; }
 
11. Set the row column values. Each fetch will be one row so you'll need to reset the indices for each "item".
 
row["typeData"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].typeData.toString(); row["value"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].value.toString();;
row["percent1"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent1.toString();;
row["percent2"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent2.toString();;
row["columnsName"] = columnsNameArray.get(innerItemsCtr);
row["linesName"] = linesNameArray.get(outerItemsCtr);
 
innerItemsCtr++;
 
if (innerItemsCtr == innerItemsLength) {
outerItemsCtr++;
innerItemsCtr = 0;
}
 
currentrow++;
return true;

Arif shah

Hi Clement,
 
Many thanks for the kind help. I really appreciate all the help that you have been providing. I have tried to use your code as below. However, it is giving an error that cannot read property "value" from undefined. For testing purpose,I have pasted the whole xml as a hardcoded string. into result. Below is the error stack
<pre class="_prettyXprint">
org.eclipse.birt.report.engine.api.EngineException: There are errors evaluating script "if( currentrow >= totalItems ){
return false;
}

row["value"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].value.toString();
row["columnsName"] = columnsNameArray.get(innerItemsCtr);
row["typeData"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].typeData.toString();
row["percent1"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent1.toString();;

row["percent2"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent2.toString();;
row["linesName"] = linesNameArray.get(outerItemsCtr);

innerItemsCtr++;

if (innerItemsCtr == innerItemsLength) {

outerItemsCtr++;

innerItemsCtr = 0;

}

currentrow++;

return true;":
Fail to execute script in function __bm_FETCH(). Source:

" + if( currentrow >= totalItems ){
return false;
}

row["value"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].value.toString();
row["columnsName"] = columnsNameArray.get(innerItemsCtr);
row["typeData"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].typeData.toString();
row["percent1"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent1.toString();;

row["percent2"] = rss.items.items.item[outerItemsCtr].item[innerItemsCtr].percent2.toString();;
row["linesName"] = linesNameArray.get(outerItemsCtr);

innerItemsCtr++;

if (innerItemsCtr == innerItemsLength) {

outerItemsCtr++;

innerItemsCtr = 0;

}

currentrow++;

return true; + "

A BIRT exception occurred. See next exception for more information.
TypeError: Cannot read property "value" from undefined (/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]#6).
   at org.eclipse.birt.report.engine.script.internal.DtEScriptExecutor.handleJS(DtEScriptExecutor.java:99)
   at org.eclipse.birt.report.engine.script.internal.DataSetScriptExecutor.handleJS(DataSetScriptExecutor.java:256)
   at org.eclipse.birt.report.engine.script.internal.ScriptDataSetScriptExecutor.handleFetch(ScriptDataSetScriptExecutor.java:143)
   at org.eclipse.birt.data.engine.impl.ScriptDataSetRuntime.fetch(ScriptDataSetRuntime.java:103)
   at org.eclipse.birt.data.engine.impl.PreparedScriptDSQuery$ScriptDSQueryExecutor$CustomDataSet.fetch(PreparedScriptDSQuery.java:260)
   at org.eclipse.birt.data.engine.executor.cache.OdiAdapter.fetch(OdiAdapter.java:226)
   at org.eclipse.birt.data.engine.executor.cache.RowResultSet.fetch(RowResultSet.java:145)
   at org.eclipse.birt.data.engine.executor.cache.RowResultSet.doNext(RowResultSet.java:118)
   at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:96)
   at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet$RowResultSetWithResultSetScope.fetch(SimpleResultSet.java:1013)
   at org.eclipse.birt.data.engine.executor.cache.RowResultSet.doNext(RowResultSet.java:118)
   at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:96)
   at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet.doNext(SimpleResultSet.java:754)
   at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet.next(SimpleResultSet.java:742)
   at org.eclipse.birt.data.engine.impl.ResultIterator.hasNextRow(ResultIterator.java:590)
   at org.eclipse.birt.data.engine.impl.ResultIterator.nextRow(ResultIterator.java:530)
   at org.eclipse.birt.data.engine.impl.ResultIterator.next(ResultIterator.java:480)
   at org.eclipse.birt.report.engine.data.dte.QueryResultSet.next(QueryResultSet.java:191)
   at org.eclipse.birt.report.engine.executor.ListingElementExecutor.hasNextChild(ListingElementExecutor.java:252)
   at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
   at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
   at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
   at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:62)
   at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
   at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:46)
   at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
   at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
   at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
   at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
   at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.resumeLayout(HTMLInlineStackingLM.java:111)
   at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.layoutNodes(HTMLInlineStackingLM.java:160)
   at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
   at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
   at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
   at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
   at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:46)
   at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
   at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
   at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:92)
   at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:100)
   at org.eclipse.birt.report.engine.presentation.ReportDocumentBuilder.build(ReportDocumentBuilder.java:249)
   at org.eclipse.birt.report.engine.api.impl.RunTask.doRun(RunTask.java:269)
   at org.eclipse.birt.report.engine.api.impl.RunTask.run(RunTask.java:86)
   at org.eclipse.birt.report.service.ReportEngineService.runReport(ReportEngineService.java:1325)
   at org.eclipse.birt.report.service.BirtViewerReportService.runReport(BirtViewerReportService.java:158)
   at org.eclipse.birt.report.service.actionhandler.BirtRunReportActionHandler.__execute(BirtRunReportActionHandler.java:81)
   at org.eclipse.birt.report.service.actionhandler.BirtGetPageActionHandler.__checkDocumentExists(BirtGetPageActionHandler.java:58)
   at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.prepareParameters(AbstractGetPageActionHandler.java:118)
   at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.__execute(AbstractGetPageActionHandler.java:103)
   at org.eclipse.birt.report.service.actionhandler.AbstractBaseActionHandler.execute(AbstractBaseActionHandler.java:90)
   at org.eclipse.birt.report.soapengine.processor.AbstractBaseDocumentProcessor.__executeAction(AbstractBaseDocumentProcessor.java:47)
   at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.executeAction(AbstractBaseComponentProcessor.java:143)
   at org.eclipse.birt.report.soapengine.processor.BirtDocumentProcessor.handleGetPage(BirtDocumentProcessor.java:87)
   at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.process(AbstractBaseComponentProcessor.java:112)
   at org.eclipse.birt.report.soapengine.endpoint.BirtSoapBindingImpl.getUpdatedObjects(BirtSoapBindingImpl.java:66)
   at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
   at java.lang.reflect.Method.invoke(Unknown Source)
   at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
   at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
   at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
   at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
   at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
   at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
   at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
   at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
   at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
   at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.doPost(BirtSoapMessageDispatcherServlet.java:265)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
   at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
   at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.service(BirtSoapMessageDispatcherServlet.java:122)
   at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
   at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
   at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
   at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:60)
   at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
   at org.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:360)
   at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
   at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
   at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
   at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
   at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
   at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   at org.eclipse.jetty.server.Server.handle(Server.java:370)
   at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
   at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
   at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
   at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
   at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
   at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
   at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
   at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
   at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   at java.lang.Thread.run(Unknown Source)
Caused by: org.eclipse.birt.data.engine.core.DataException: Fail to execute script in function __bm_FETCH(). Source:

A BIRT exception occurred. See next exception for more information.
TypeError: Cannot read property "value" from undefined (/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]#6)
 at org.eclipse.birt.data.engine.script.JSMethodRunner.runScript(JSMethodRunner.java:147)
 at org.eclipse.birt.report.engine.script.internal.DtEScriptExecutor.handleJS(DtEScriptExecutor.java:90)
... 101 more
Caused by: org.eclipse.birt.data.engine.core.DataException: A BIRT exception occurred. See next exception for more information.
TypeError: Cannot read property "value" from undefined (/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]#6)
 at org.eclipse.birt.data.engine.core.DataException.wrap(DataException.java:123)
 at org.eclipse.birt.data.engine.script.ScriptEvalUtil.evaluateJSAsExpr(ScriptEvalUtil.java:997)
 at org.eclipse.birt.data.engine.script.JSMethodRunner.runScript(JSMethodRunner.java:138)
... 102 more
Caused by: org.eclipse.birt.core.exception.CoreException: TypeError: Cannot read property "value" from undefined (/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]#6)
 at org.eclipse.birt.core.script.JavascriptEvalUtil.wrapRhinoException(JavascriptEvalUtil.java:307)
 at org.eclipse.birt.core.script.JavascriptEvalUtil.evaluateRawScript(JavascriptEvalUtil.java:106)
 at org.eclipse.birt.core.script.JavascriptEvalUtil.evaluateScript(JavascriptEvalUtil.java:138)
 at org.eclipse.birt.data.engine.script.ScriptEvalUtil.evaluateJSAsExpr(ScriptEvalUtil.java:993)
... 103 more
Caused by: org.mozilla.javascript.EcmaError: TypeError: Cannot read property "value" from undefined (/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]#6)
 at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3654)
 at org.mozilla.javascript.ScriptRuntime.constructError(ScriptRuntime.java:3632)
 at org.mozilla.javascript.ScriptRuntime.typeError(ScriptRuntime.java:3660)
 at org.mozilla.javascript.ScriptRuntime.typeError2(ScriptRuntime.java:3679)
 at org.mozilla.javascript.ScriptRuntime.undefReadError(ScriptRuntime.java:3692)
 at org.mozilla.javascript.ScriptRuntime.getObjectProp(ScriptRuntime.java:1421)
 at org.mozilla.javascript.gen.c130._c1(/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"]:6)
 at org.mozilla.javascript.gen.c130.call(/report/data-sets/script-data-set[@id="230"]/method[@name="fetch"])
 at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:108)
 at org.mozilla.javascript.gen.c10._c0(/report/data-sets/script-data-set[@id="21"]/method[@name="fetch"]:0)
 at org.mozilla.javascript.gen.c10.call(/report/data-sets/script-data-set[@id="21"]/method[@name="fetch"])
 at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:398)
 at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3065)
 at org.mozilla.javascript.gen.c10.call(/report/data-sets/script-data-set[@id="21"]/method[@name="fetch"])
 at org.mozilla.javascript.gen.c10.exec(/report/data-sets/script-data-set[@id="21"]/method[@name="fetch"])
 at org.eclipse.birt.core.script.JavascriptEvalUtil.evaluateRawScript(JavascriptEvalUtil.java:99)
... 105 moreorg.eclipse.birt.report.engine.api.EngineException: Data Set script method "Fetch" returned null; expected a Boolean value.
 at org.eclipse.birt.report.engine.script.internal.ScriptExecutor.addException(ScriptExecutor.java:249)
 at org.eclipse.birt.report.engine.script.internal.ScriptExecutor.addException(ScriptExecutor.java:238)
 at org.eclipse.birt.report.engine.script.internal.ScriptDataSetScriptExecutor.handleFetch(ScriptDataSetScriptExecutor.java:165)
 at org.eclipse.birt.data.engine.impl.ScriptDataSetRuntime.fetch(ScriptDataSetRuntime.java:103)
 at org.eclipse.birt.data.engine.impl.PreparedScriptDSQuery$ScriptDSQueryExecutor$CustomDataSet.fetch(PreparedScriptDSQuery.java:260)
 at org.eclipse.birt.data.engine.executor.cache.OdiAdapter.fetch(OdiAdapter.java:226)
 at org.eclipse.birt.data.engine.executor.cache.RowResultSet.fetch(RowResultSet.java:145)
 at org.eclipse.birt.data.engine.executor.cache.RowResultSet.doNext(RowResultSet.java:118)
 at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:96)
 at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet$RowResultSetWithResultSetScope.fetch(SimpleResultSet.java:1013)
 at org.eclipse.birt.data.engine.executor.cache.RowResultSet.doNext(RowResultSet.java:118)
 at org.eclipse.birt.data.engine.executor.cache.RowResultSet.next(RowResultSet.java:96)
 at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet.doNext(SimpleResultSet.java:754)
 at org.eclipse.birt.data.engine.executor.transform.SimpleResultSet.next(SimpleResultSet.java:742)
 at org.eclipse.birt.data.engine.impl.ResultIterator.hasNextRow(ResultIterator.java:590)
 at org.eclipse.birt.data.engine.impl.ResultIterator.nextRow(ResultIterator.java:530)
 at org.eclipse.birt.data.engine.impl.ResultIterator.next(ResultIterator.java:480)
 at org.eclipse.birt.report.engine.data.dte.QueryResultSet.next(QueryResultSet.java:191)
 at org.eclipse.birt.report.engine.executor.ListingElementExecutor.hasNextChild(ListingElementExecutor.java:252)
 at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
 at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
 at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportItemExecutor.hasNextChild(WrappedReportItemExecutor.java:86)
 at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:62)
 at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
 at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:46)
 at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
 at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
 at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
 at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
 at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.resumeLayout(HTMLInlineStackingLM.java:111)
 at org.eclipse.birt.report.engine.layout.html.HTMLInlineStackingLM.layoutNodes(HTMLInlineStackingLM.java:160)
 at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
 at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
 at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
 at org.eclipse.birt.report.engine.layout.html.HTMLStackingLM.layoutChildren(HTMLStackingLM.java:26)
 at org.eclipse.birt.report.engine.layout.html.HTMLRepeatHeaderLM.layoutChildren(HTMLRepeatHeaderLM.java:46)
 at org.eclipse.birt.report.engine.layout.html.HTMLAbstractLM.layout(HTMLAbstractLM.java:140)
 at org.eclipse.birt.report.engine.layout.html.HTMLBlockStackingLM.layoutNodes(HTMLBlockStackingLM.java:70)
 at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:92)
 at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:100)
 at org.eclipse.birt.report.engine.presentation.ReportDocumentBuilder.build(ReportDocumentBuilder.java:249)
 at org.eclipse.birt.report.engine.api.impl.RunTask.doRun(RunTask.java:269)
 at org.eclipse.birt.report.engine.api.impl.RunTask.run(RunTask.java:86)
 at org.eclipse.birt.report.service.ReportEngineService.runReport(ReportEngineService.java:1325)
 at org.eclipse.birt.report.service.BirtViewerReportService.runReport(BirtViewerReportService.java:158)
 at org.eclipse.birt.report.service.actionhandler.BirtRunReportActionHandler.__execute(BirtRunReportActionHandler.java:81)
 at org.eclipse.birt.report.service.actionhandler.BirtGetPageActionHandler.__checkDocumentExists(BirtGetPageActionHandler.java:58)
 at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.prepareParameters(AbstractGetPageActionHandler.java:118)
 at org.eclipse.birt.report.service.actionhandler.AbstractGetPageActionHandler.__execute(AbstractGetPageActionHandler.java:103)
 at org.eclipse.birt.report.service.actionhandler.AbstractBaseActionHandler.execute(AbstractBaseActionHandler.java:90)
 at org.eclipse.birt.report.soapengine.processor.AbstractBaseDocumentProcessor.__executeAction(AbstractBaseDocumentProcessor.java:47)
 at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.executeAction(AbstractBaseComponentProcessor.java:143)
 at org.eclipse.birt.report.soapengine.processor.BirtDocumentProcessor.handleGetPage(BirtDocumentProcessor.java:87)
 at sun.reflect.GeneratedMethodAccessor87.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
 at org.eclipse.birt.report.soapengine.processor.AbstractBaseComponentProcessor.process(AbstractBaseComponentProcessor.java:112)
 at org.eclipse.birt.report.soapengine.endpoint.BirtSoapBindingImpl.getUpdatedObjects(BirtSoapBindingImpl.java:66)
 at sun.reflect.GeneratedMethodAccessor86.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
 at java.lang.reflect.Method.invoke(Unknown Source)
 at org.apache.axis.providers.java.RPCProvider.invokeMethod(RPCProvider.java:397)
 at org.apache.axis.providers.java.RPCProvider.processMessage(RPCProvider.java:186)
 at org.apache.axis.providers.java.JavaProvider.invoke(JavaProvider.java:323)
 at org.apache.axis.strategies.InvocationStrategy.visit(InvocationStrategy.java:32)
 at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
 at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
 at org.apache.axis.handlers.soap.SOAPService.invoke(SOAPService.java:454)
 at org.apache.axis.server.AxisServer.invoke(AxisServer.java:281)
 at org.apache.axis.transport.http.AxisServlet.doPost(AxisServlet.java:699)
 at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.doPost(BirtSoapMessageDispatcherServlet.java:265)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
 at org.apache.axis.transport.http.AxisServletBase.service(AxisServletBase.java:327)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
 at org.eclipse.birt.report.servlet.BirtSoapMessageDispatcherServlet.service(BirtSoapMessageDispatcherServlet.java:122)
 at org.eclipse.equinox.http.registry.internal.ServletManager$ServletWrapper.service(ServletManager.java:180)
 at org.eclipse.equinox.http.servlet.internal.ServletRegistration.service(ServletRegistration.java:61)
 at org.eclipse.equinox.http.servlet.internal.ProxyServlet.processAlias(ProxyServlet.java:128)
 at org.eclipse.equinox.http.servlet.internal.ProxyServlet.service(ProxyServlet.java:60)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
 at org.eclipse.equinox.http.jetty.internal.HttpServerManager$InternalHttpServiceServlet.service(HttpServerManager.java:360)
 at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:684)
 at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503)
 at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
 at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
 at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:429)
 at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
 at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
 at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
 at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
 at org.eclipse.jetty.server.Server.handle(Server.java:370)
 at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:494)
 at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:982)
 at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1043)
 at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
 at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
 at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
 at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:696)
 at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:53)
 at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
 at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
 at java.lang.Thread.run(Unknown Source)
Caused by: org.eclipse.birt.data.engine.core.DataException: Data Set script method "Fetch" returned null; expected a Boolean value.
 at org.eclipse.birt.report.engine.script.internal.ScriptDataSetScriptExecutor.handleFetch(ScriptDataSetScriptExecutor.java:153)
... 99 more

</pre>

Clement Wong

I'm not sure if you have defined the Output Columns in your Scripted Data Set. Did you define the 6 columns such as "value" or "percent1"?
 
It would be best to attach your Report Design for further analysis.

Arif shah

Yes, I have defined all of them. Here is a sample rptdesign file
 
Regards
Syed

Clement Wong

I made a few corrections and also I realized that we don't need a Java ArrayList and we can just access the columnsName and linesName directly with the various indexes.
 
Attached is a working solution.

Arif shah

Hi Clement,
 
This is perfect. Many thanks for the kind help. Just one last question. In actual implementation, I am first running another dataset (that uses webservice type data source.) which calls the webservice and then stores the returned xml in a pgv. I am then using that pgv in our result variable
 
If I wanted to call the webservice directly from within this scripted datasource to store the returned xml in the result variable, can you please guide me how would that work.I have no idea how to call a webservice from a scripted datasource
 
Once again, many many thanks for the kind support. It has been a great help
 
Regards

Clement Wong

Glad to help.
 
Regarding calling a web service from a Scripted Data Source, there are various posts and DevShares on that topic. In particular, did you review my previously mentioned DevShare @ <a data-ipb='nomediaparse' href='http://developer.actuate.com/community/forum/index.php?/files/file/1122-parsing-xml-in-birt-with-e4x? '>http://developer.actuate.com/community/forum/index.php?/files/file/1122-parsing-xml-in-birt-with-e4x? </a>The web service in that example is a ESPN RSS feed, and the code you'll want to look at is in the beginning of the beforeOpen event in the Scripted Data Set.

Arif shah

Many thanks for replying to my questions. yes, that code was very helpful for basic understanding but I am struggling with one thing. The web request that you are making uses REST while in my case, it uses SOAP. So it sends an xml request as well as retrieves an xml response. In your example, it seems to be uisng a simple URL. I have no idea how to adjust the code for sending the xml/SOAP request.

Clement Wong

There are many ways to send a SOAP request in BIRT using Java APIs such as <a data-ipb='nomediaparse' href='http://docs.oracle.com/javase/7/docs/api/javax/xml/soap/package-summary.html'>jaxax.xml.soap</a> or Apache Axis. If you can do it in Java, you can do it in BIRT.

Arif shah

I have the following example from java that sends an xml soap request to web and retrieves the response but I dont know how and where to use it in birt in my example. Could you please help me send my soap request using the below example and my earlier file that sends a soap request to web
 
I would then like to use the returned xml string from the below request and use your earlier example to convert them into rows and columns. Many thanks
<pre class="_prettyXprint">
import javax.xml.soap.*;
import javax.xml.transform.*;
import javax.xml.transform.stream.*;

public class SOAPClientSAAJ {

/**
* Starting point for the SAAJ - SOAP Client Testing
*/
public static void main(String args[]) {
try {
// Create SOAP Connection
SOAPConnectionFactory soapConnectionFactory = SOAPConnectionFactory.newInstance();
SOAPConnection soapConnection = soapConnectionFactory.createConnection();

// Send SOAP Message to SOAP Server
String url = "http://ws.cdyne.com/emailverify/Emailvernotestemail.asmx";
SOAPMessage soapResponse = soapConnection.call(createSOAPRequest(), url);

// Process the SOAP Response
printSOAPResponse(soapResponse);

soapConnection.close();
} catch (Exception e) {
System.err.println("Error occurred while sending SOAP Request to Server");
e.printStackTrace();
}
}

private static SOAPMessage createSOAPRequest() throws Exception {
MessageFactory messageFactory = MessageFactory.newInstance();
SOAPMessage soapMessage = messageFactory.createMessage();
SOAPPart soapPart = soapMessage.getSOAPPart();

String serverURI = "http://ws.cdyne.com/";

// SOAP Envelope
SOAPEnvelope envelope = soapPart.getEnvelope();
envelope.addNamespaceDeclaration("example", serverURI);

/*
Constructed SOAP Request Message:
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:example="http://ws.cdyne.com/">
<SOAP-ENV:Header/>
<SOAP-ENV:Body>
<example:VerifyEmail>
<example:email>mutantninja@gmail.com</example:email>
<example:LicenseKey>123</example:LicenseKey>
</example:VerifyEmail>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
*/

// SOAP Body
SOAPBody soapBody = envelope.getBody();
SOAPElement soapBodyElem = soapBody.addChildElement("VerifyEmail", "example");
SOAPElement soapBodyElem1 = soapBodyElem.addChildElement("email", "example");
soapBodyElem1.addTextNode("mutantninja@gmail.com");
SOAPElement soapBodyElem2 = soapBodyElem.addChildElement("LicenseKey", "example");
soapBodyElem2.addTextNode("123");

MimeHeaders headers = soapMessage.getMimeHeaders();
headers.addHeader("SOAPAction", serverURI + "VerifyEmail");

soapMessage.saveChanges();

/* Print the request message */
System.out.print("Request SOAP Message = ");
soapMessage.writeTo(System.out);
System.out.println();

return soapMessage;
}

/**
* Method used to print the SOAP Response
*/
private static void printSOAPResponse(SOAPMessage soapResponse) throws Exception {
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = transformerFactory.newTransformer();
Source sourceContent = soapResponse.getSOAPPart().getContent();
System.out.print("\nResponse SOAP Message = ");
StreamResult result = new StreamResult(System.out);
transformer.transform(sourceContent, result);
}

}</pre>

Clement Wong

I have cited and provided a few samples already. Typically, you would perform this type of operation in the beforeOpen event of your Scripted Data Source.