Download content via webservice problem - Out of Memory

Hi all,

We have problems with downloading of content via web service. Any content which is bigger that 30-35 Mb requires a lot of heap space to can be downloaded. For example, to download a file of 100 mb, the used heap space increased to 1550 Mb. We use MTOM feature for Content Service also, and I went to the temporary files and noticed that the content is complete downloaded in the temporary file, but exactly when the temporary file is finished, the heap increase and crash. Also, we used jProfiler to monitors the JVM, and we noticed that all the memory is occupied by arrays of char[]. We use OpenText Content Server 10.5.0, but we have the same problem also with 10.0.0, 9.7.1 and 16.0.0.

The stacktrace looks like this:

java.lang.OutOfMemoryError: Java heap space
   at java.util.Arrays.copyOf(Arrays.java:2367)
   at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
   at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:114)
   at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:415)
   at java.lang.StringBuilder.append(StringBuilder.java:132)
   at com.sun.org.apache.xerces.internal.dom.TextImpl.getWholeText(TextImpl.java:168)
   at com.sun.xml.ws.streaming.DOMStreamReader.next(DOMStreamReader.java:757)
   at com.sun.xml.bind.v2.runtime.unmarshaller.StAXStreamConnector.bridge(StAXStreamConnector.java:192)
   at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:360)
   at com.sun.xml.bind.v2.runtime.BridgeImpl.unmarshal(BridgeImpl.java:120)
   at com.sun.xml.bind.api.Bridge.unmarshal(Bridge.java:233)
   at com.sun.xml.ws.client.sei.ResponseBuilder$DocLit.readResponse(ResponseBuilder.java:525)
   at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:121)
   at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:89)
   at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:118)
   at com.sun.proxy.$Proxy37.downloadContent(Unknown Source)
   at de.fme.mc.otcs.downloader.Downloader.run(Downloader.java:93)
   at de.fme.mc.otcs.downloader.AppStarter.main(AppStarter.java:17)

We used also both services, the java web service and .net web service, but same result. Are there some configurations in OpenText Content Server or anything related with that?

We tried to use the HttpContentHandler.ashx from .net web service and works better, but for files bigger than ~70 Mb, a lot of bytes was lost.

Any ideas? Thanks

Find more posts tagged with

Comments

Appu Nair

I am wondering if you saw this from Hugh( and Ossie) do they serve better?

https://knowledge.opentext.com/knowledge/llisapi.dll?func=ll&objId=63003361&objAction=viewincontainer&ShowReplyEntry=63012285#forum_topic_63012285

vgavrila

Thank you for response. I saw that solution and I used it, but the content is not always downloaded completly and we can not use it in this way. We are wondering why the service requires so much java heap space.

vgavrila

We found the solution. The method with HttpContentHandler.ashx is the good one, but the troubles was caused by IIS configurations. In web.config of .net webservices you have to set some parameters like follow:

<requestLimits maxAllowedContentLength="1610612736" /> (represents bytes)

If you have the content very large you have to set the concerned value in settings That is it! All the best!

HFCDev

Dumb question, but where in the web.config do you set those parameters?

-Hugh

appuGmail

Hugh,

I usually go to the Web.Config in the IIS application that points to the CWS or les-services from OT.Actually we don.t directly use the webservices folder in the <OTHOME> we copy it locally to another place in the IIS location and change many things.

Here's such a file for your refer. This was tested and put there so a webservices vendor((not OT) could get up to 4 GB of files in livelink

I suggest you compare it with a standard Web.Config that comes from OT and look through the changes.All these changes can be found documented

by MicroSoft.

I think I have put <!--Appu on where all I changed from OT and I think I had to make ContentService wait longer as in time outs.

I dont know whether the ashx handler uses these settings but it did work to their satisfaction.

https://knowledge.opentext.com/knowledge/cs.dll/Properties/66778755

If developers had pulled a App.Config before the changes to Web.Config was made if they refresh the connection it will reflect the server setup.

That is how I prove that webservices can upload more.If IIS fails over the upload then TC OOB will be able to do more.

..... appnair appunair( Livelink Rocks )

Well, if I called the wrongnumber, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

On Sat, Mar 11, 2017 at 8:00 AM, eLink Entry: Content Web Services Forum <otdncontentwebservices971forum@elinkkc.opentext.com> wrote:

Download content via webservice problem - Out of Memory Posted byFerguson, HughOn 03/11/2017 08:56 AM

Dumb question, but where in the web.config do you set those parameters?-Hugh
[To post a comment, use the normal reply function]
Topic: Download content via webservice problem - Out of Memory
Forum: Content Web Services Forum
Content Server: Knowledge Center CS16