Document Upload using REST API fails in CS 16.2.7+ 4

cameronl@hayesis.com.au

Hi all,

I have attached the standard type of REST testing HTML/Javascript file with the four functions (Authenticate, Get User Details, Create Folder, Upload Document).

On two different sites on their Production CS 16.2.4 systems, all four functions work.

On these two different sites on their Test 16.2.8 systems, the first three work but "Upload Document" gives a 500 Internal Server error.

On a third site on their Production CS 16.2.7 system, the first three work, but "Upload Document" gives a 500 Internal Server error. (Test is off line at the moment)

Has the REST API commands to upload a document changed between 16.2.4 and 16.2.7+ ?

Find more posts tagged with

Comments

cameronl@hayesis.com.au

Trying to upload Attachment....

Content Server Error: An illegal attempt was made to upload a document

Return to Wizard

Quit Wizard

cameronl@hayesis.com.au

Upload html as text

Content Server Error: Could not invoke the method.

Return to Wizard

Quit Wizard

Kyle_Swidrovich

The API has not changed in that respect.
A 500 error would imply that your next troubleshooting step should be evaluating what's happening on the server side.
Do you get any additional details in your 500 error response?
If not, you may need to start reviewing the Content Server thread logs for the failing request.

Matthew_Pinkney

Is your upload directory set correctly? (?func=admin.sysvars) . That needs to be set to an appropriate value for uploads to work reliably.

cameronl@hayesis.com.au

Thanks Kyle.

I was seeing "ERRORWebDocUtils.GetFileUploadInfo: An illegal attempt was made to upload a document" but now I seem to be seeing "INFO Dispatcher error: Could not process object, invalid action 'create'".

cameronl@hayesis.com.au

I have the Upload Directory set to "\\collut1\transfer\",

04/26/2019 00:20:24 INFO LLSYSPARAMS_ASSOC = 'A<1,?,''_uploadFilenames''={},''_uploadPath''=''\\\\collut1\\transfer\\''>'

I am uploading from that location.

04/26/2019 00:20:24 INFO file = '\\collut1\transfer\CamUpload.txt'

But I am still getting an error

04/26/2019 00:20:24 INFO WebDocUtils.GetFileUploadInfo: Using file: \\collut1\transfer\CamUpload.txt
04/26/2019 00:20:24 ERROR WebDocUtils.GetFileUploadInfo: An illegal attempt was made to upload a document - \\collut1\transfer\CamUpload.txt

When I try to upload the txt thread log file here as an attachment I get

=================

Content Server Error: An illegal attempt was made to upload a document
Return to Wizard Quit Wizard

================

So here it is in the body

===========================================

04/26/2019 00:20:24 INFO Fri Apr 26 00:20:24 2019
04/26/2019 00:20:24 INFO _REQUEST = 'llweb'
04/26/2019 00:20:24 INFO AUTH_TYPE = ''
04/26/2019 00:20:24 INFO CONTENT_LENGTH = '118'
04/26/2019 00:20:24 INFO CONTENT_TYPE = 'application/x-www-form-urlencoded; charset=UTF-8'
04/26/2019 00:20:24 INFO description = 'Desc'
04/26/2019 00:20:24 INFO file = '\\collut1\transfer\CamUpload.txt'
04/26/2019 00:20:24 INFO GATEWAY_INTERFACE = 'CGI/1.1'
04/26/2019 00:20:24 INFO HTTP_ACCEPT = 'application/json, text/javascript, */*; q=0.01'
04/26/2019 00:20:24 INFO HTTP_ACCEPT_ENCODING = 'gzip, deflate'
04/26/2019 00:20:24 INFO HTTP_ACCEPT_LANGUAGE = 'en-US,en;q=0.9'
04/26/2019 00:20:24 INFO HTTP_CONNECTION = 'keep-alive'
04/26/2019 00:20:24 INFO HTTP_CONTENT_LENGTH = '118'
04/26/2019 00:20:24 INFO HTTP_CONTENT_TYPE = 'application/x-www-form-urlencoded; charset=UTF-8'
04/26/2019 00:20:24 INFO HTTP_HOST = 'uatlfe1'
04/26/2019 00:20:24 INFO HTTP_ORIGIN = 'null'
04/26/2019 00:20:24 INFO HTTP_OTCSTICKET = '58Qx5WMy0hdQsWj9oSGXTr8kP8lY9xlwE7iiR5b/MMYbrFruQ3NjpANtPyCwTPzyZrPGKzni06Qu0n4xHZnbPWC6tfsBrRo2'
04/26/2019 00:20:24 INFO HTTP_USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36'
04/26/2019 00:20:24 INFO HTTPS = 'off'
04/26/2019 00:20:24 INFO HTTPS_KEYSIZE = ''
04/26/2019 00:20:24 INFO HTTPS_SECRETKEYSIZE = ''
04/26/2019 00:20:24 INFO HTTPS_SERVER_ISSUER = ''
04/26/2019 00:20:24 INFO HTTPS_SERVER_SUBJECT = ''
04/26/2019 00:20:24 INFO LLSYSPARAMS_ASSOC = 'A<1,?,''_uploadFilenames''={},''_uploadPath''=''\\\\collut1\\transfer\\''>'
04/26/2019 00:20:24 INFO name = 'REST Upload Test 3 doc'
04/26/2019 00:20:24 INFO parent_id = '13703162'
04/26/2019 00:20:24 INFO PATH_INFO = '/api/v2/nodes'
04/26/2019 00:20:24 INFO PATH_TRANSLATED = 'C:\inetpub\wwwroot\api\v2\nodes'
04/26/2019 00:20:24 INFO QUERY_STRING = ''
04/26/2019 00:20:24 INFO REMOTE_ADDR = '10.254.83.172'
04/26/2019 00:20:24 INFO REMOTE_HOST = '10.254.83.172'
04/26/2019 00:20:24 INFO REMOTE_USER = ''
04/26/2019 00:20:24 INFO REQUEST_ID = '07ea43b6-6776-11e9-8706-005056af41fc'
04/26/2019 00:20:24 INFO REQUEST_METHOD = 'POST'
04/26/2019 00:20:24 INFO REQUEST_PROCESSING_DURATION = '0'
04/26/2019 00:20:24 INFO SCRIPT_NAME = '/livelink/livelink.exe'
04/26/2019 00:20:24 INFO SERVER_NAME = 'uatlfe1'
04/26/2019 00:20:24 INFO SERVER_PORT = '80'
04/26/2019 00:20:24 INFO SERVER_PROTOCOL = 'HTTP/1.1'
04/26/2019 00:20:24 INFO SERVER_SOFTWARE = 'Microsoft-IIS/10.0'
04/26/2019 00:20:24 INFO type = '144'
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: Profiler
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: GlobalServerErrorCondition
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() server=[UATLDB1]
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() cached connect uses log =[connect1.log]
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() return cached connect using connect1.log.
04/26/2019 00:20:24 DEBUG called CreateNewNamed for we32518 - not cached
04/26/2019 00:20:24 DEBUG #Warning: Setting value on non-temp in R/O OSpace: #550004db.fCache = A<1,?>
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() server=[UATLDB1]
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() cached connect uses log =[connect1.log]
04/26/2019 00:20:24 DEBUG CAPIAllocConnect() return cached connect using connect1.log.
04/26/2019 00:20:24 DEBUG KFilePrefs::GetKFilePrefs - sharing loaded pref: \OPENTEXT\config\opentext.ini
04/26/2019 00:20:24 DEBUG dbOptionRecompile is set to FALSE.
04/26/2019 00:20:24 DEBUG KFilePrefs::GetKFilePrefs - sharing loaded pref: \OPENTEXT\config\opentext.ini
04/26/2019 00:20:24 DEBUG KFilePrefs::GetKFilePrefs - sharing loaded pref: \OPENTEXT\config\opentext.ini
04/26/2019 00:20:24 DEBUG LLIAPI::DBConnect.ResetObject(): Resetting the DBConnect object.
04/26/2019 00:20:24 DEBUG LLIAPI::DBConnect.ResetObject(): Reset cached rights data.
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: OTDSValidateTicketResult
04/26/2019 00:20:24 INFO Check HTTP Referer has been disabled for the RestAPI since it does not make sense.
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: OTDSSettingsLoaded
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: OTDSValidateTicketResult
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: ServiceString
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: KernelValidationTempDir
04/26/2019 00:20:24 INFO needs validation: \OPENTEXT\temp\webdoc\
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: KernelValidationTempDir
04/26/2019 00:20:24 INFO WebDocUtils.GetFileUploadInfo: Using file: \\collut1\transfer\CamUpload.txt
04/26/2019 00:20:24 ERROR WebDocUtils.GetFileUploadInfo: An illegal attempt was made to upload a document - \\collut1\transfer\CamUpload.txt
04/26/2019 00:20:24 DEBUG KFilePrefs::GetKFilePrefs - sharing loaded pref: \OPENTEXT\config\opentext.ini
04/26/2019 00:20:24 DEBUG GetGlobal - No such global: Builder

cameronl@hayesis.com.au

NB. The form calling REST via javascript is just for testing.

A third party app that interfaced with CS 16.2.4 system and used REST to upload documents stopped working in the Test environment when it was upgraded to CS 16.2.7. The Test environment was rolled back to 16.2.4 and the third party app started to successfully upload to CS again.

A month later the Test environment was then upgraded from CS 16.2.4 to CS 16.2.8 and third party app stopped working again.

I implemented the form calling REST via javascript to prove it was not an issue with the CS16.2.8 upgrade, but looks like it was.

Matthew_Pinkney

File upload security improved around that time. specifying file=c:\temp\myfile.txt will not work but using a html page/ajax with a file picker or your favorite rest client and a file picker will.

Matthew_Pinkney

By file picker I mean an input of type file on an html page and then using ajax to post it
HTML
<input id="myfile" name="myfile" type="file">
JS
var file = document.getElementById("myfile").files[0];
formData.append("file",file)

Or using the file option in your rest client instead of specifying a string.

cameronl@hayesis.com.au

I'll try that.

Matthew, could you please try attaching a file to a Reply to this post and see if you get the same error as I do

"Content Server Error: An illegal attempt was made to upload a document"

cameronl@hayesis.com.au

Interesting.

If I revert the contents of the 16.2.8 webdoc folder ("D:\OPENTEXT\core\module\webdoc") to the contents of the 16.2.4 version (D:\OPENTEXT\module\webdoc_16_2_0) then the upload via REST is successful.

Switch it back to the 16.2.8 version and the illegal error comes back.

Matthew_Pinkney

Hi Cameron,

I can reproduce the adding of an attachment issue you're seeing in the forums. This should be fixed in 16.2.9 by that team.

Regarding the REST API, the upload API has not changed. Using html/javascript should work similar if you use something like what was indicated in my previous post.
Passing a file path like c:\temp\myfile.txt to the REST is not supported.

Matt

Steve Bauman

Are there any plans of adding support outside of JavaScript uploads to the API?

I'm building a PHP based application and using cURL to interact with the OpenText CS API and it would be amazing if this support was added. I can't do this via client side JavaScript - we need a background process to send documents into the Content Server as it's a bulk upload. We would need to enforce users to sign into the server via the API themselves prior to upload the documents, which we definitely want to avoid.

cameronl@hayesis.com.au

Determined that the issue is caused by a change in 16.2.8 GetFileUploadInfo fuction in "D:\OPENTEXT\core\module\webdoc\ospace\webdoc.oll"

If I comment out this section it all works again.

Q: How to parameterise the REST BODYTEXT (not FORM) call so that this new check does not fail ?

  //
  // If the request came through the CGI, ensure that the file also came through the CGI.
  //
  if ( ok && IsDefined( uploadFilenames ) )

if ( versionFile in uploadFilenames == 0 )

    ok = FALSE
    echoError( Str.Format( "%1.GetFileUploadInfo: An illegal attempt was made to upload a document - %2", OS.Name( this ), versionFile ) )
    errMsg = [WEB_ERRMSG.AnIllegalAttempWasMadeToUploadADocument]
    versionFile = ''

end

cameronl@hayesis.com.au

When using the Advanced REST Client (ARC) app for testing does anyone know how to simulate an attach dialog box (to specify the file to upload) as specifying the file as a parameter is not allowed ?

appuGmail

That to me looks like it is checking if the upload directory is filled therefore it knows the file is not being sent through perhaps like an API.

Filling the Upload directory with a valid value usually breaks Object Importer and I have seen this particular script the root cause of it

but typically OT support standard answer is in a cluster of servers keep a few with that empty to get OI loads in . In the OI instruction file, one

is typically putting a <File > </File> which is a UNC path a String representation. My guess is OT is thinking since its a controlled process one may

not be susceptible to a hacking/fishing attempt. But what you are saying is the REST API used to accept this and it might have gotten into OT's security wing this is a possible security hack like the Cookie not being able to be passed in.

I would through a support ticket ask OT what is the reason and explain a genuine case where you find their approach unacceptable as an API perhaps that will reach Development and more pertinent eyes would look at it.

As for how would you patch it may be very easy to defeat that in a patch but again since I do not know the security implications of that I wouldn't bother to try to post a workaround. I would probably in a controlled setting see if not having the

upload directory filled gets you by.

Ciao,

..... appnair appunair( Livelink Rocks )

Well, if I called the wrongnumber, why did you answer the phone?
James Thurber, New Yorker cartoon caption, June 5, 1937

On Fri, Apr 26, 2019 at 11:59 PM eLink Entry: Content Server Development Forum <development@elinkkc.opentext.com> wrote:

Document Upload using REST API fails in CS 16.2.7+ 4 4 Posted bycameronl@hayesis.com.au (Lyon, Cameron)On 04/27/2019 12:56 AM

Determined that the issue is caused by a change in 16.2.8 GetFileUploadInfo fuction in "D:\OPENTEXT\core\module\webdoc\ospace\webdoc.oll"If I comment out this section it all works again.Q: How to parameterise the REST BODYTEXT (not FORM) call so that this new check does not fail ?  //
  // If the request came through the CGI, ensure that the file also came through the CGI.
  //
  if ( ok && IsDefined( uploadFilenames ) )
   if ( versionFile in uploadFilenames == 0 )
    ok = FALSE
    echoError( Str.Format( "%1.GetFileUploadInfo: An illegal attempt was made to upload a document - %2", OS.Name( this ), versionFile ) )
    errMsg = [WEB_ERRMSG.AnIllegalAttempWasMadeToUploadADocument]
    versionFile = ''
   end
  end
[To post a comment, use the normal reply function]
Topic: Document Upload using REST API fails in CS 16.2.7+ 4
Forum: Content Server Development Forum
Content Server: My Support

Kyle_Swidrovich

The client used to upload a file isn’t the issue.

The Content Server web client (CGI/LLISAPI/Servlet) examines the MIME parts of the inbound request payload, and extracts any files it detects to the upload directory, referencing each file in the uploadFilenames list.

A supported file upload will use Content-Disposition: form-data with afilename attribute.

For example:

Content-Disposition: form-data; name="file"; filename="sampleimage.png"

Content-Type: image/png

What do your payloads look like?

Kyle

cameronl@hayesis.com.au

04/29/2019 10:51:38 INFO _REQUEST = 'llweb'
04/29/2019 10:51:38 INFO AUTH_TYPE = ''
04/29/2019 10:51:38 INFO CONTENT_LENGTH = '100'
04/29/2019 10:51:38 INFO CONTENT_TYPE = 'application/x-www-form-urlencoded'
04/29/2019 10:51:38 INFO filename = '\\hgcsid02\upload\CamUpload1.txt'
04/29/2019 10:51:38 INFO GATEWAY_INTERFACE = 'CGI/1.1'
04/29/2019 10:51:38 INFO HTTP_CONTENT_LENGTH = '100'
04/29/2019 10:51:38 INFO HTTP_CONTENT_TYPE = 'application/x-www-form-urlencoded'
04/29/2019 10:51:38 INFO HTTP_HOST = 'hgcsid02'
04/29/2019 10:51:38 INFO HTTP_OTCSTICKET = '40bASqZoRV2rHveZNATaN0GX1lH2+Oue1ycnwCESjEuPMDJC/k1+2ukDkNPvTT4ydOtoJLn+QYD9P2W8vp86gqUd3/xW6H3O'
04/29/2019 10:51:38 INFO HTTPS = 'off'
04/29/2019 10:51:38 INFO HTTPS_KEYSIZE = ''
04/29/2019 10:51:38 INFO HTTPS_SECRETKEYSIZE = ''
04/29/2019 10:51:38 INFO HTTPS_SERVER_ISSUER = ''
04/29/2019 10:51:38 INFO HTTPS_SERVER_SUBJECT = ''
04/29/2019 10:51:38 INFO LLSYSPARAMS_ASSOC = 'A<1,?,''_uploadFilenames''={},''_uploadPath''=''\\\\hgcsid02\\upload\\''>'
04/29/2019 10:51:38 INFO name = 'REST Upload Test f INt doc'
04/29/2019 10:51:38 INFO parent_id = '1995615'
04/29/2019 10:51:38 INFO PATH_INFO = '/api/v2/nodes'
04/29/2019 10:51:38 INFO PATH_TRANSLATED = 'C:\inetpub\wwwroot\api\v2\nodes'
04/29/2019 10:51:38 INFO QUERY_STRING = ''
04/29/2019 10:51:38 INFO REMOTE_ADDR = '172.22.86.76'
04/29/2019 10:51:38 INFO REMOTE_HOST = '172.22.86.76'
04/29/2019 10:51:38 INFO REMOTE_USER = ''
04/29/2019 10:51:38 INFO REQUEST_ID = 'b5dd0ba6-6a29-11e9-942f-005056923b33'
04/29/2019 10:51:38 INFO REQUEST_METHOD = 'POST'
04/29/2019 10:51:38 INFO REQUEST_PROCESSING_DURATION = '0'
04/29/2019 10:51:38 INFO SCRIPT_NAME = '/otcs/cs.exe'
04/29/2019 10:51:38 INFO SERVER_NAME = 'hgcsid02'
04/29/2019 10:51:38 INFO SERVER_PORT = '80'
04/29/2019 10:51:38 INFO SERVER_PROTOCOL = 'HTTP/1.1'
04/29/2019 10:51:38 INFO SERVER_SOFTWARE = 'Microsoft-IIS/8.5'
04/29/2019 10:51:38 INFO type = '144'

cameronl@hayesis.com.au

Hi Kyle,

PLease see Ticket 4145743.

I have uploaded the HTML/Javascript utility and screenshots of the REST call settings and parameters I have used in the Chrome Extension Advanced REST Client

Matthew_Pinkney

I think you may need to adjust the way you are uploading, it will still work with non JS. I am able to use cURL to upload a file like the following from a cmd prompt in windows
curl "<my server goes here>/cs162/cs.exe/api/v2/nodes" ^
-H "<your auth goes here>" ^
-F "name=imageExample" ^
-F "type=144" ^
-F "parent_id=2000" ^
-F "file=@C:\OPENTEXT\cs162upload\image.jpg"

Matthew_Pinkney

RE: Choosing a file in a REST client.
In RESTlet client for chrome you can chose file type for a parameter instead of text.
In Advanced REST Client, switch to multipart/form-data and click "add file part" under the body section. Select the file.

Pierre N

Also working on file upload in REST API on CS1628

it looks like an ARC upload request works with a multipart/form-data content and a file field named "file"

if the file field is named anything other than "file" I get an

{

"error": "An illegal attempt was made to upload a document"

}

Kyle_Swidrovich

The file parameter requirement is outlined in the documentation, under POST api/v2/nodes:

https://knowledge.opentext.com/knowledge/llisapi.dll?func=ll&objId=67789539&objAction=browse&viewType=1

Kyle

From: eLink Entry: Content Server Development Forum <development@elinkkc.opentext.com>
Sent: Tuesday, April 30, 2019 1:11 PM
To: eLink Recipient <devnull@elinkkc.opentext.com>
Subject: RE Document Upload using REST API fails in CS 16.2.7+ 4 4

RE Document Upload using REST API fails in CS 16.2.7+ 4 4

Posted bypnocera@globalcents.com (Nocera, Pierre) On 04/30/2019 01:08 PM

Also working on file upload in REST API on CS1628

it looks like an ARC upload request works with a multipart/form-data content and a file field named "file"

if the file field is named anything other than "file" I get an

{

"error": "An illegal attempt was made to upload a document"

}

[To post a comment, use the normal reply function]
Topic:	Document Upload using REST API fails in CS 16.2.7+ 4
Forum:	Content Server Development Forum
Content Server:	My Support

cameronl@hayesis.com.au

Hi Kyle, Matthew et al.

Our applications are written to to post the payload as "Content-Type", "application/x-www-form-urlencoded" instead of "Content-Type", "multipart/form-data".

If I use POSTMAN or ARC to simulate this then "Content-Type", "multipart/form-data" works on 16.2.7 and 16.2.8.

If I swap to "Content-Type", "application/x-www-form-urlencoded" it works with 16.2.7 and fails with 16.2.8 - "An illegal attempt was made to upload a document".

Tim_Coleman

I don't know how one can upload a file using an application/x-www-form-urlencoded. Doesn't it have to be multipart/form-data?

cameronl@hayesis.com.au

Using "application/x-www-form-urlencoded" works up to 16.2.7, and although "multipart/form-data" is the preferred method when uploading a file it is a lot easier for a simpleton like me (and others) to code for "application/x-www-form-urlencoded".
But it appears the changes to webdoc.oll in 16.2.8 have ended the use "application/x-www-form-urlencoded" to upload files, so I am going to have to tell my client they will have to rewrite their apps before we can go to 16.2.8

Tim_Coleman

What do you mean it works? You encode the file content into the form data?

cameronl@hayesis.com.au

This works in 16.2.7 using Content-Type of "application/x-www-form-urlencoded"

    strUrl = "http://[Servername]/livelink/livelink.exe/api/v2/nodes"
    strPayload = "type=144&parent_id=14752908&name=RESTUploadTestC3doc&file=\\PC1060\upload\camupload2.txt"
    blnAsync = False

    Call Get_Ticket

    With objRequest
        .Open "POST", strUrl, blnAsync
        .SetRequestHeader "Content-Type", "application/x-www-form-urlencoded"
        .SetRequestHeader "OTCSTICKET", strOTCSTicket
        .Send strPayload

        'spin wheels whilst waiting for response
        While objRequest.readyState <> 4
            DoEvents
        Wend
        strResponse = .ResponseText
    End With

Tim_Coleman

But how do you upload the file content? Are you saying the file already exists on the server?

cameronl@hayesis.com.au

I don't do anything else.

As far as I knew CS uploaded the file from the specified location directly just like Object Importer does.

I run that command and the specified file is uploaded.

Cameron Lyon - Technical Services Manager (IM)

HAYES Information Systems

On 4 May 2019, at 6:57 am, eLink Entry: Content Server Development Forum <development@elinkkc.opentext.com> wrote:

eLink : RE RE Document Upload using REST API fails in CS 16.2.7+ 4 4
RE RE Document Upload using REST API fails in CS 16.2.7+ 4 4 Posted by tcoleman@opentext.com (Coleman, Tim) On 05/03/2019 06:49 PM

But how do you upload the file content? Are you saying the file already exists on the server?
[To post a comment, use the normal reply function]
Topic: Document Upload using REST API fails in CS 16.2.7+ 4
Forum: Content Server Development Forum
Content Server: My Support