Exception loading large documents using LLTreeView

Hello,

I am getting the exception trace below when attempting to upload large documents (> 3.5 MB) to livelink using LLTreeView. Looking at other threads it seems that the issue is related to a bug in using MTOM in transferring large files. Is this the case here ?

Appreciate your help. Thank you.

Amir Sharobim.

javax.xml.ws.WebServiceException: java.io.IOException: Error writing to server

at com.sun.xml.ws.transport.http.client.HttpClientTransport.checkResponseCode(HttpClientTransport.java:258)

at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:151)

at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:88)

at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:595)

at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:554)

at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:539)

at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:436)

at com.sun.xml.ws.client.Stub.process(Stub.java:248)

at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:134)

at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:244)

at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:224)

at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:117)

at $Proxy41.uploadContent(Unknown Source)

at aenvsrd.ecm.ws.ECMUtils.uploadDocument(ECMUtils.java:264)

at aenvsrd.ecm.ws.ECMPerformanceTest.testUploadDocuments(ECMPerformanceTest.java:66)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)

at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)

at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)

at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)

at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)

at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:66)

at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)

at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)

at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)

at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)

at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:38)

at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)

Caused by: java.io.IOException: Error writing to server

at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:438)

at sun.net.www.protocol.http.HttpURLConnection.writeRequests(HttpURLConnection.java:450)

at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:998)

at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:373)

at com.sun.xml.ws.transport.http.client.HttpClientTransport.checkResponseCode(HttpClientTransport.java:221)

... 34 more

Find more posts tagged with

Comments

sgrasley

Hi, Amir.

LLTreeView does not use MTOM (it uses the GetVersionContents method rather than GetVersionContentsContext and DownloadContent), which is probably why you're getting the exception. Are you getting an OutOfMemoryError on the server?

thanks,

Scott

amir333

Hi Scott,

I thought the client is using the MTOM method all along. That is good news. I will try it with MTOM.

Thanks for the reply.

Amir Sharobim

amir333

Scott,

I looked at the TreeView java code and it is using the same MTOM method that is described in the client guide documentation. Below is sample code from the Java TreeView

(as you can see it is using the createDocumentContext along with uploadContent which is what the client guide has described when using MTOM). At the end of the message i have also included an excerpt from the log file.

Please advise.

Thanks

Amir

in the addDocumentButtonActionPerformed method in Tree.java it has :

...

// get a contextID representing the cached meta-data for the upload

				contextID = fDocMan.createDocumentContext( parentID, uploadFile.getName(), "Enterprise Web Services Demo", false, null );

				// start a thread to do the upload of the contents

				fFileUploadTask = new FileUploadTask( contextID, uploadFile, fileAtts );

				fFileUploadThread = new Thread( null, fFileUploadTask, "FileUploadTask" );

				fFileUploadThread.start();

...

and in the FileUploadTask in Tree.java it is using:

...

// JAX-WS doesn't handle the headers automatically, need to manually add

				WebServiceUtil.setSoapHeader( (WSBindingProvider) fContent, fOTAuth, fContextID, fFileAtts );

				fBytesProcessed = 0;

				fObjectID = fContent.uploadContent( new DataHandler( new FileDataSource( fUploadFile ) ) );

				fBytesProcessed = fUploadFile.length();

...

Here is an excerpt from the log file when the 6.7 MB was being uploaded and as you can see it ended abruptly:

2009-09-24 09:29:19,3430 TRACE [3] BasicServiceContext(): fServer = VNN106, fServerPort = 2199, fServerEncoding = DEFAULT

2009-09-24 09:29:19,3586 TRACE [3] Service()

2009-09-24 09:29:19,3586 TRACE [3]   IServiceContext = BasicServiceContext[fServer=VNN106,fServerPort=2199,fServerEncoding=DEFAULT]

2009-09-24 09:29:19,3586 TRACE [3]   IRequestContext = BasicRequestContext[fClientIPAddress=199.215.66.34,fServiceToken=,]

2009-09-24 09:29:19,3586 TRACE [3]   IRequestContext.ServiceToken = mMRmQ9l1hwrEfkKFYFiTxyJCWlz7%2FHXM4UQmZQ2j1FY%3D

2009-09-24 09:29:19,3586 TRACE [3] begin: DocumentManagement.CreateDocumentContext(188950,6.7MBfile.zip,Enterprise Web Services Demo,False,)

2009-09-24 09:29:19,4211 TRACE [3] end:   DocumentManagement.CreateDocumentContext()

2009-09-24 09:29:19,4211 TRACE [3]   IRequestContext.ServiceToken =

sgrasley

Sorry, Amir, I was looking at the code for downloading. You're right, MTOM is used for downloading. However, the connection object will still try and buffer the file as it downloads it, so an extra line of code is required to prevent it from reading the entire file into memory. Add the following line to WebServiceUtil's getContentService() method:

// instruct the connections to use chunked transfer encoding to support large files
( (BindingProvider) endpoint ).getRequestContext().put( JAXWSProperties.HTTP_CLIENT_STREAMING_CHUNK_SIZE, 1024000 );

Add it before the call to setSoapHeader().

Thanks, and sorry again for the confusion.

Scott

amir333

Hi Scott,

I have added the line as you indicated but now i am getting a bad request exception. Am i using the wrong jar file ?

com.sun.xml.ws.client.ClientTransportException: The server sent HTTP status code 400: Bad Request

	at com.sun.xml.ws.transport.http.client.HttpClientTransport.checkResponseCode(HttpClientTransport.java:252)

	at com.sun.xml.ws.transport.http.client.HttpTransportPipe.process(HttpTransportPipe.java:151)

	at com.sun.xml.ws.transport.http.client.HttpTransportPipe.processRequest(HttpTransportPipe.java:88)

	at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:595)

	at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:554)

	at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:539)

	at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:436)

	at com.sun.xml.ws.client.Stub.process(Stub.java:248)

	at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:134)

	at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:244)

	at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:224)

	at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:117)

	at $Proxy41.uploadContent(Unknown Source)

	at aenvsrd.ecm.ws.ECMUtils.uploadDocument(ECMUtils.java:269)

	at aenvsrd.ecm.ws.ECMPerformanceTest.testUploadDocuments(ECMPerformanceTest.java:63)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

	at java.lang.reflect.Method.invoke(Method.java:597)

	at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)

	at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)

	at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)

	at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)

	at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)

	at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:66)

	at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)

	at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)

	at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)

	at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)

	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:38)

	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)

sgrasley

Hi, Amir.

Do you have the web services deployed on IIS? If so, try adding this to the web services' Web.config file:

<system.web>
<httpRuntime maxRequestLength="65536"/>
</system.web>

Thanks,

Scott

amir333

You are THE man, Scott. That just did it.

Error messages are misleading ....

I was able to upload up to 160 MB.

Had to increase the java heap size to 1 GB, increase the maxRequestLength to 2 GB and play with the HTTP_CLIENT_STREAMING_CHUNK_SIZE value.

i am trying to see how much larger will i be able to load.

Thanks for all of your help and time on this forum. VERY VERY helpful.

Amir Sharobim.

sgrasley

Thanks for posting on the forum, Amir. Please don't hesitate to post any other questions, comments, interesting findings, etc.

Thanks,

Scott