nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Search API revisited - CS16 and up

HFCDev

Hi,

I'm helping a client with a proof-of-concept. Among the many features that need to be proven is the ability to programmatically search. For what we're doing, I'm working primarily with the WS API (for one thing, I need to be able to demonstrate the handling of really big, i.e. up to 10 GB files, and I don't think REST can handle that). It sounds like the search WS API is not a viable option and clients are being pushed to using either the XML API (which appears to be wrapping calls to the web UI) or REST. My questions thus are as follows:

* What is the status of search WS API? Is the recommendation from OT still to not use this if possible?

* If I did need to do search by either XML API or REST, is there a way I can share authentication credentials between the two methods? If I can't have retrieval and search share the same API type, I'd like them to at least share an authentication token.

* Finally, are any of the programmatic approaches to search able to return either a summary or a thumbnail? In the POC, we will want to show users skimming through search results and finding the image or summary text, depending on whether the hit is a document or a drawing.

Thanks in advance

-Hugh Ferguson

Find more posts tagged with

Comments

AKropp

Hi Hugh,

I can probably speak to some of this.

* What is the status of search WS API? Is the recommendation from OT still to not use this if possible?

-Try to avoid it.

-As of 10.5 2015-09 the XML API will accept a REST token for auth.

-I don’t think any of our APIs expose thumbnails at all. Summary can definitely returned though. What you get back is beholden to either your template or what specific things you ask for in the query, since it can get pretty complex. But given a basic system default template and a simple search you’ll see it returned by region name OTSummary as an XML element, or as simply “Summary” in the JSON data.

Sidenote: A colleague of mine internally successfully transferred an enormous file – somewhere around 13-16GB in size – via CS REST API. IIS won’t let you do it since it has some built-in hard limits at the 4GB mark, but it looks like if you’re on Tomcat then the sky is the limit more or less.

From: eLink Entry: Content Web Services Forum [mailto:otdncontentwebservices971forum@elinkkc.opentext.com]
Sent: Thursday, February 23, 2017 9:48 AM
To: eLink Recipient <devnull@elinkkc.opentext.com>
Subject: Search API revisited - CS16 and up

Search API revisited - CS16 and up

Posted by Ferguson, Hugh On 02/23/2017 09:36 AM

Hi,

* What is the status of search WS API? Is the recommendation from OT still to not use this if possible?

Thanks in advance

-Hugh Ferguson

[To post a comment, use the normal reply function]
Forum:	Content Web Services Forum
Content Server:	Knowledge Center CS16

HFCDev

Hi Alex,

Thanks for the response. So it sounds like the search WS is a no-go and I probably should use REST. I just went over to developer.opentext.com to look up the V2 version of the API. For uploading a document, I haven't found a method to set the category attributes on the object you're uploading either during the upload/create operation or after the fact. The Category API's method to get actions available on a particular node might be an option but I haven't found a working example yet. That being said, I can upload a document and apply metadata using WS API and I have that working. Is the auth token used in WSAPI transferrable to REST?

Thanks

-Hugh

AKropp

Good question – I’ve never thought to actually try that.

To that end I quickly fired up VS and nabbed a Web Services auth token, then I crammed that value into a general nodes GET on the REST side and it worked fine. This was against 10.5 2016-12 I happened to have running at this time. Only real “gotcha” looks like the WS token comes URL-encoded, so I also quickly decoded it before I pasted it into Postman. Of course this may not be necessary if one is doing this “properly” (I made an assumption that Postman would URL-encode whatever values I pasted in the otcsticket header which would mess it up).

At least as far as this quick test goes it seems to work just fine. Good to know!

Cheers,

From: eLink Entry: Content Web Services Forum [mailto:otdncontentwebservices971forum@elinkkc.opentext.com]
Sent: Thursday, February 23, 2017 10:52 AM
To: eLink Recipient <devnull@elinkkc.opentext.com>
Subject: RE Search API revisited - CS16 and up

RE Search API revisited - CS16 and up

Posted by Ferguson, Hugh On 02/23/2017 10:39 AM

Hi Alex,

Thanks

-Hugh

[To post a comment, use the normal reply function]
Topic:	Search API revisited - CS16 and up
Forum:	Content Web Services Forum
Content Server:	Knowledge Center CS16

HFCDev

Quoted Alex Kropp on 02/23/2017 01:02 PM:

I'd like to know more about how to do this. I can deploy Tomcat in our POC environment. In the past, using WS to upload/download, I was hitting a limit in the MB area (going against IIS set to the maximum of 2 Gig) and ended up using the non-MTOM approach with the HttpContentHandler.ashx WS which is only available in IIS. That begs the question, what's the equivalent in a Tomcat environment (so I can get around 4 G limit)?

My preference would be to handle file transfer with WS and use REST/XML for search (noticed that search REST does support thumbnails). I need to be able to demonstrate a 10 G upload and download from a client app. In the example of using the non-MTOM approach in the WS documentation, they reference the HTTP Servlet - is this the SEA servlet?

-Hugh

Dave_Kinchlea

Hi Hugh

We regularly upload >>4GB files using CWS 10.5 and other than ensuring Tomcat is the web server (still Windows though) providing the service, there was nothing special we had to do. We use the following C# code (after setting up the System.IO.FileStream):

// create the document context
string createdDocContext = documentManagement.CreateDocumentContext(ref authenticationDM,
parentId,
name,
comments,
advVersionControl,
otMetadata);
// now add the content and complete the document
string createdDocId = contentService.UploadContent(ref authenticationCS, createdDocContext, fileAtts, dataStream);

I believe we hit a limit at 17GB though we didn't pursue it because we have placed our own limit at 10GB for the application.

Quoted Hugh Ferguson on 02/23/2017 12:43 PM:

-Hugh

Ferdinand Prantl

Reusing the SOAP WS token for the REST API is a side-effect of the current implementation, which can change anytime. I have been saying this since CS 10, so you can guess, what is the probabilty of such change :-) If you want to authenticate by a single token type to any API, you should request the token from OTDS.

You need to make an extra request for the particular node (v1/nodes/:id/thumbnails/medium/content) to get its thumbnail. If you need to retrieve mutliple thumbnails by a single call, you could write a custom REST call for it, or you could wait until all of them are loaded and then show them. If I imagine 30 search results with their thumbnails randomly popping up, once each one gets loaded in the browser, it probably would not make a professional UI look & feel :-)

If you want to upload a file of a size greater than 2GB to a CS with IIS, you would have to write your own HTTP module and UI for it. It is easier getting another web server running as an alternative, which does not have this limitation. CS and REST API do not limit the file size. You may hit other limits, though. Database, file system, or your patience ;-) Another hard limit would be probably 64-bit file length, but it may be problem first for your children :-)

If you want to set categories (or other not built-in node properties) during the node creation, you need to pass them within their role. For example, uploading a document with categories:

function upload(file) {  var body = {        type: 144,        parent_id: 19051,        name: file.name,        roles: {          categories: {            {              '52271': { // category                '52271_2': 'test' // attribute              }            }          }        }      },      formData = new FormData();  formData.append('body', JSON.stringify(body));  formData.append('file', file);  return $      .ajax({        type: 'POST',        url: '.../api/v1/nodes',        data: formData,        headers: {otcsticket: otcsticket}      })      .then(function (response) {        ... succeeded      }, function (jqxhr) {        ... failed      });}

HFCDev

Hi Kinch,

have you ever coded the client in Java? I'm asking because I hit a much, much lower limit with Java (there's a post where I describe the Out of memory error I was getting). It was Ossie Moore who got me onto using the HttpContentHandler (in IIS WS) rather than relying on the existing JAX stubs. I don't recall the exact reason why I had to do this for Java, but this approach won't work here because IIS has an upper limit of 4GB for all files, and 2 GB for calls through ASP.NET which is what the WS API is using. I can deploy Tomcat, but it appears I can't do the equivalent of calling HttpContentHandler.ashx in Tomcat.

@OpenText, do the Java client stubs (when generated via WSImport) for CS 16 have any limits as they seem to have done in CS 10.0?

Thanks

-Hugh

HFCDev

Hi Ferdinand,

If this works, I may use the REST for everything and just write a Java layer around it. I'm mentioning java alot but this is client imposed. Quickly though, to use this approach, obviously I need Tomcat as IIS will **** out at 2-4 GB. In an existing IIS environment, is it enough to just follow the installation instructions for the Content Server servlet to be able to use REST through Tomcat?

Thanks

-Hugh

Dave_Kinchlea

No, sorry, I don't do JAVA personally and we are using C# for all our client code though it did look like we might have to port to JAVA at first, turns out that the only thing we had to watch out for was that "INT" in dotnet WS is 32-bit and in Java WS is 64-bit so there was a minor amount of porting to assure we could access the data objects properly. I can't help you wrt JAVA client I'm afraid.

I will say that I did a small bit of digging and will retract the "17GB limit" observation ... while our application should have rejected it, there is a 56GB file in our system today that was put there using CWS (though exactly who is going to download and access such a beast remains a mystery :-)). I will assume the limit is dependent up the resources provided (temp disk space, RAM, etc).

Quoted Hugh Ferguson on 02/23/2017 01:31 PM:

Hi Kinch,

@OpenText, do the Java client stubs (when generated via WSImport) for CS 16 have any limits as they seem to have done in CS 10.0?

Thanks

-Hugh

AKropp

Hi Hugh,

You don’t even need to do that much. Once Tomcat is installed just deploy the WAR file for the services (aka copy the cws.war into the webapps directory and wait a sec). If Tomcat is on the same host as a CS instance it should work fine as-is, otherwise tweak the web.xml config file to point to a CS instance hostname and service port and restart. It’s actually surprisingly easy to set it up.

From: eLink Entry: Content Web Services Forum [mailto:otdncontentwebservices971forum@elinkkc.opentext.com]
Sent: Thursday, February 23, 2017 3:22 PM
To: eLink Recipient <devnull@elinkkc.opentext.com>
Subject: RE Search API revisited - CS16 and up

RE Search API revisited - CS16 and up

Posted by Ferguson, Hugh On 02/23/2017 03:18 PM

Hi Ferdinand,

Thanks

-Hugh

[To post a comment, use the normal reply function]
Topic:	Search API revisited - CS16 and up
Forum:	Content Web Services Forum
Content Server:	Knowledge Center CS16

AKropp

Oops, disregard my previous message. Misread that as deploying the WS API in a mixed-webserver environment. Not so sure about the CS REST requirements for same.

From: Alex Kropp
Sent: Thursday, February 23, 2017 3:55 PM
To: 'NjYxNDQxNjUA@elinkkc.opentext.com' <NjYxNDQxNjUA@elinkkc.opentext.com>; eLink Recipient <devnull@elinkkc.opentext.com>
Subject: RE: RE Search API revisited - CS16 and up

Hi Hugh,

RE Search API revisited - CS16 and up

Posted by Ferguson, Hugh On 02/23/2017 03:18 PM

Hi Ferdinand,

Thanks

-Hugh

[To post a comment, use the normal reply function]
Topic:	Search API revisited - CS16 and up
Forum:	Content Web Services Forum
Content Server:	Knowledge Center CS16

HFCDev

Following up on search after spending some time on upload/download....

Quoted Alex Kropp on 03/06/2017 02:12 PM:

-As of 10.5 2015-09 the XML API will accept a REST token for auth.

Can you or anyone give an example of passing the REST token in a request to the search API? Do we formulate the header the same way we do for REST? The programming examples I've found all assume that you're trying to populate the cookie store by invoking ll.login first. that just seems kind of clumsy especially as I'm already getting at least a REST otcs token and possibly one for CWS (depending on what functionality I need).

Thanks

-Hugh

Pramod Mohandas

Hi Hugh,

Take a look at Alex’s response in this post. I had bookmarked it recently for the exact purpose. He has a code snippet attached in one of the response which showcases exactly what you’re looking for.

https://knowledge.opentext.com/knowledge/llisapi.dll?func=ll&objId=63736237&objAction=viewincontainer&ShowReplyEntry=63749615#forum_topic_63749615

TLDR; You essentially to pass in the REST token as part of the header in your search requests as a value to the otcsticket.

HFCDev

Found it, thanks!

Also, I managed to get the authentication of XML Search to work with a WS token, although I had to URL Decode it first.

-Hugh

HFCDev

Quoted Ferdinand Prantl on 05/02/2017 01:58 PM:

If you want to set categories (or other not built-in node properties) during the node creation, you need to pass them within their role. For example, uploading a document with categories:

function upload(file) {  var body = {        type: 144,        parent_id: 19051,        name: file.name,        roles: {          categories: {            {              '52271': { // category                '52271_2': 'test' // attribute              }            }          }        }      },      formData = new FormData();  formData.append('body', JSON.stringify(body));  formData.append('file', file);  return $      .ajax({        type: 'POST',        url: '.../api/v1/nodes',        data: formData,        headers: {otcsticket: otcsticket}      })      .then(function (response) {        ... succeeded      }, function (jqxhr) {        ... failed      });}

I am just revisiting this, and I want to look at embedding category attribute data into a REST upload request. I get what the numbers are - the first number (in this case 52271) is the category ID and the second number (the _2 in 52271_2) is the attribute ID, but if I'm writing a wrapper class for developers familiar with another leading application and not Content Server, I would ideally like them to be working with category and attribute names. To that end, I would need to translate the category name into a DAPINODE ID, which is easy, but I can't seem to find the correct REST call to give me the ID's of attributes in a given category. I checked out developer.opentext.com but the API seems to get me information about categories on an object, not the generalized category definition.

And finally, if I was composing a REST request with metadata embedded, what is the correct format if the attribute is a date field?

Thanks

-Hugh

Ferdinand Prantl

About referencing a category

A category is a node and can be stored in any volume. The only unique identifer of it is its node ID. If you want to let the user to choose a category by its name, you need to show some selector, probably browsing one or more locations, where you expect the categories to be stored. CS UI Widgets offer NodePicker for this, for example. As a result, you get the node ID of the category.

About getting information about a category

If you want to get the category schema, which includes IDs, titles, types and other metadata of category attributes, you can use the same CS REST calls, which CS Smart UI uses for this. The key is, that you do not ask about the category in general, but you ask about it in the scope of the scenario, which you implement. It can be either creating a new node (/api/v1/forms/nodes/create), editing an existing one (/api/v1/forms/nodes/update), copying (/api/v1/forms/nodes/copy), or moving (/api/v1/forms/nodes/move).

1. You need information about categories, which a document will get assigned, once you create it in a specific parent container:

GET .../api/v1/forms/nodes/create?parent_id=2000&type=144{  "forms": [    {      "data": {        "1509470": {          "1509470_5": null,          ...        }      },      "options": {...},      "role_name": "categories",      "schema": {        "properties": {          "1509470": {            "properties": {              "1509470_5": {                "title": "Age",                "type": "integer",                "required": false,                ...              },              ...            },            "title": "Person",            "type": "object"          }        },        "title": "Categories",        "type": "object"      }    }  ]}

Categories are represented as sub-forms of a single form, which you can find in the "forms" array by (role_name === 'categories').

You can learn more about JSON Schema and Alpaca extensions at:

2. You need information about a category, which you want to assign to a document in addition to the inherited ones, once you create it in a specific parent container:

GET .../api/v1/forms/nodes/create?parent_id=2000&type=144&category_id=123456

The output of this call is exactly the same, only including the additional category "123456".

About the date field

CS REST API uses a modified ISO 8601 format - without milliseconds and without the time zone. For example, instead of "2017-05-05T14:10:52.740Z" you send (or receive) "2017-05-05T14:10:52". If you want to send a date-time value (for category attribute with the time part, for example), you have to send it in the server time zone. If you want to send a date-only value (for category attribute without the time part, for example), you send it as midnight: "2017-05-05T00:00:00".

HFCDev

I just worked this out. First off, there is a small error in this JSON

Quoted Hugh Ferguson on 05/09/2017 03:21 PM:

function upload(file) {  var body = {        type: 144,        parent_id: 19051,        name: file.name,        roles: {          categories: {            {              '52271': { // category                '52271_2': 'test' // attribute              }            }          }        }      },

In the above, if you want to embed the categories, leave out the '52271' { ...category json } and just make that category json which will be a list of catID_attrID name/value pairs.

here is an example of what my JSON looked like:

{"parent_id":10022,
"roles":{"categories":{"20005_3":"2017-05-09T15:12:49",

"20005_2":"123400042",

"20005_9":"CSUI_SDK_Intro (1).pptx"}

},
"name":"CSUI_SDK_Intro (1).pptx",

"type":144}'

-Hugh

HFCDev

Also, to convert category and attribute names into their respective ID's, I leveraged the Search XML API since I already built a Java project around it, and it successfully was able to integrate an existing call to get the category region map which gave me the info I needed (plus a bunch more about the attributes in case I wanted to do validation on the inputs).

having the equivalent as a REST call would be preferable as JSON is more lightweight than XML.

In the meantime, Web Services does provide all the functionality I want, I merely experimented with trying to reproduce as much of it in REST in case the client needs to extend the API to provide additional functionality which I don't think anyone will disagree, is much easier to do in REST than in WS.

-Hugh

Ferdinand Prantl

I do not know, what extensions are planned to the current /search REST call. You should not expect, that XML Search will be "converted" to REST soon. The usage scenario is different.

CWS are the preferred B2B API. Server-to-server, thick-client-to-server communication. You can enjoy the generated parameter types and method proxy-stubs to provide sanity checks ont only in the API layer, but also in the compiler. You would have code too much to achieve the same safety of communication using the REST API. There are activities to describe REST interfaces with metadata, so that the same automation can be achieved, like with SOAP WS, but they have not become a standard. And they kind of contradict the purpose of the REST - be simple by not requiring the bulk of SOAP.

REST API is preferred for the thin-client-to-server communication. Because the platform is not as rich as the native OS, you should go the same way. For example, you should implement an expensive protocol in JavaScript. You sacrifice the safety checks in the SOAP protocol layer on the client side, because you handle the errors in a general way - show a message to the user and use an integrated debugging/logging environment like the web browser developer tools. For example, you use just HTTP status and divide the errors to "missing object" (404), "unauthorized access" (403), "invalid action" (400), "server problem" (500) and other numbers.

Yes, creating a REST API resource is easier, than creating a SOAP WS service/method, mainly because you can stay in OScript only. But it is the current situation. It is not because REST is different from SOAP. The CS team might decide to improve it.