nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Polling a folder continuously

Kapil_Chaudhari

Hi Scott and others,

We have a requirement where documents are scanned and sent to the library folder. We will have to keep polling the library folder continuously to see if a new document has arrived. If there is a new document in the library folder with the desired owner id, we want to get the metadata of that document like node id, owner id etc and insert into the table.

How can we poll the library folder continuously?

Thanks,

Kapil

Find more posts tagged with

Comments

Appu_Nair

There are many ways what Ican think of now

Assign a specific notification on the folder and give a generic email and you will receive an email for item s added.You will hav the nodeid in the report.You can use lapi/webservices/oscript to get metadata.
With livelink SDK you can create a customization that sends you the requiiste data on folder additions /update.I beleive when things get added,nodeuPdate callback may get fired havenot tested this.This is oscript work.
Using regular SQL you can look for childitem count of the folder(top and recursion)
You can write oscipt customization to send a workflow .E scan uses this methodology as an option.You might be able to look at what Escan and DP does and see if you can code this with Microsoft Jscript.

sgrasley

To build on Appu's response, you can use DocumentManagement.GetNode() on the folder to determine the child count. This will only give you the immediate children, though, so as Appu said you may have to recurse into subfolders.

You might also be able to use Search to accomplish what you're looking for. You could restrict the results to items created after a certain date and within a certain location.

Neither of these are continuous, but might be useful.

Thanks,

Scott

Kapil_Chaudhari

Thanks Appu and Scott. I will try out these and keep the thread updated.

Kapil_Chaudhari

Hi Scott,

Can you please give me an example of the Search service to achieve this? How can we build the SOAP message to see if new documents were added to a certail folder on that day? I am little stuck in finding this answer. I am leaning away from the database approach because the table will be huge in the future and the performance might be bad in coming years as more and more people starts using the application.

I also tried the notification approach. It doesnt let me configure the email text and doesn't have node_id in it.

Thanks,

Kapil

sgrasley

Hi, Kapil.

Please see the search sample pinned in this forum for an example of using SearchService.

Using a query like the following might do the trick:

where1=[qlregion OTLocation] FOLDER_ID&lookfor1=complexquery&boolean2=and&where2=[qlregion OTCreateDate] YYYYMMDD&lookfor2=complexquery

where FOLDER_ID is the ID of your folder and YYYYMMDD is the date, in that format.

Thanks,

Scott

Kapil_Chaudhari

Thanks Scott. I got it working to some extent. I used your query and also referred the pinned example. I have couple of questions. I am trying to retrieve documents which were created on a particular day. It returns the list of all the documents in that folder instead of only returning the valid ones. Can you please check if I am doing something wrong?

My request was as follows:

<soapenv:Body>
      <urn1:Search>

         <urn1:queries>

            <urn1:DataCollectionSpec>"LES Enterprise"</urn1:DataCollectionSpec>
            <urn1:FirstResultToRetrieve>1</urn1:FirstResultToRetrieve>
            <urn1:NumResultsToRetrieve>10</urn1:NumResultsToRetrieve>

            <urn1:QueryLanguage>Livelink Search API V1</urn1:QueryLanguage>

            <urn1:ResultOrderSpec>sortByRegion=OTCreateDate</urn1:ResultOrderSpec>

            <urn1:ResultSetSpec>where1=[qlregion OTLocation] 503232&where2=[qlregion OTCreateDate] 20100721</urn1:ResultSetSpec>
            
            <urn1:ResultTransformationSpec> OTName</urn1:ResultTransformationSpec>

How can I display more attributes of the documents in the response? I tried adding one more TransformationSpec with OTCreateDate which didn't work.

         </urn1:queries>

         <urn1:partitionName>NONE</urn1:partitionName>   --I am not sure what is this,that is why i just put NONE.
      </urn1:Search>
   </soapenv:Body>

Thanks,

Kapil

sgrasley

Hi, Kapil.

The following portion from the query I suggested is missing:

&lookfor1=complexquery&boolean2=and&lookfor2=complexquery

Thanks,

Scott

sgrasley

Also, partitionName won't be used in your case, so you can leave it blank or use NONE, whatever you'd like.

Thanks,

Scott

sgrasley

Hi, Kapil.

To return more regions, just add more ResultTransformationSpec elements:

<ns2:ResultTransformationSpec>OTDataID</ns2:ResultTransformationSpec><ns2:ResultTransformationSpec>OTName</ns2:ResultTransformationSpec><ns2:ResultTransformationSpec>OTObjectSize</ns2:ResultTransformationSpec><ns2:ResultTransformationSpec>OTModifyDate</ns2:ResultTransformationSpec>

These regions must be marked as displayable in the Region Map for the Search Manager you are searching against.

Thanks,

Scott

Kapil_Chaudhari

Thanks Scott. It is working now.

Kapil

Kapil_Chaudhari

Hi Scott,

What will be the date format if I want to search the document based on date and time?

I tried YYYYMMDDhhmmSS, but it didn't work.

Thanks,

Kapil

sgrasley

Hi, Kapil.

There's a separate region called OTCreateTime that stores the time in HHmmss format, I believe. You'll likely have to mark it as queryable (and displayable if you want to return it) in the Search Manager's Region Map.

Thanks,

Scott

Kapil_Chaudhari

Hi Scott,

What is the validaty of the security token? For how long, we can reuse it?

Thanks,

Kapil

sgrasley

Hi, Kapil.

The authentication token is governed by the same restrictions as the authentication cookie for the UI. This is configurable through the admin pages under Server Configuration | Configure Security Parameters (func=admin.securityvars).

Thanks,

Scott

Kapil_Chaudhari

Quoted Scott Grasley (sgrasley) on 2010/09/01 11:02:

Hi, Kapil.

Thanks,

Scott

Hi Scott,

I've marked the OTCreateTime and OTModifyTime as queryable in search manager. How do I need to change my query to return all the documents which has OTModifyTime > some time. This is the query which returns all the documents created on 20100901. I would like to change the query in such a way that it only returns documents which were created on 20100901 and after 130000(time).

where1=[qlregion OTLocation] 579756&lookfor1=complexquery&boolean2=and&where2=[qlregion OTCreateDate] 20100901&lookfor2=complexquery

Kapil

sgrasley

Hi, Kapil.

I believe this will work (but I haven't tried it):

where1=[qlregion OTLocation] 579756&lookfor1=complexquery&boolean2=and&where2=[qlregion OTCreateDate] 20100901&lookfor2=complexquery&boolean3=and&where3=[qlregion OTCreateTime] > 130000&lookfor3=complexquery

Thanks,

Scott

Kapil_Chaudhari

Thanks Scott. The query is working. But I still can't get anything else to display in the response. I only get OTName in the response.

Here is my part of the request and I am trying get OTName,OTCreateTime and OTModifyTime in the response.

<urn1:ResultOrderSpec>sortByRegion=OTCreateDate</urn1:ResultOrderSpec>
            
            <urn1:ResultSetSpec>where1=[qlregion OTLocation] 579756&lookfor1=complexquery&boolean2=and&where2=[qlregion OTCreateDate] 20100901&lookfor2=complexquery&boolean3=and&where3=[qlregion OTModifyTime] > 103100&lookfor3=complexquery</urn1:ResultSetSpec>
            
            <urn1:ResultTransformationSpec>OTName</urn1:ResultTransformationSpec>
     <urn1:ResultTransformationSpec>OTModifyTime</urn1:ResultTransformationSpec>

But, the response only has OTName. OTname and OTModifyTime has exact same attributes turned on in search admin.

What do you think I might be missing?

Here is the part of the response:

<SearchResult>
            <ResultAnalysis>
               <ID>Map from Region Names to Display Names</ID>
               <N>
                  <S>Name</S>
                  <Type>MRD</Type>
               </N>
            </ResultAnalysis>
            <Results>
               <Item>
                  <ID>DataId=579756&Version=0</ID>
                  <N>
                     <ID>n1</ID>
                     <S>vcash Test</S>   --- Folder name
                     <Type>t1</Type>
                  </N>
               </Item>
               <Item>
                  <ID>DataId=578337&Version=1</ID>
                  <N>
                     <ID>n1</ID>
                     <S>small_document.pdf</S>
                     <Type>t1</Type>
                  </N>
               </Item>
               <ListDescription>
                  <ActualCount>2</ActualCount>
                  <IncludeCount>2</IncludeCount>
                  <ListHead>1</ListHead>
               </ListDescription>
               <Type>
                  <ID>t1</ID>
                  <Strings>OTName</Strings>
               </Type>
            </Results>
            <Type>
               <ID>MRD</ID>
               <Strings>OTName</Strings>
            </Type>
         </SearchResult>

Thanks,

Kapil

sgrasley

Hi, Kapil.

The only thing that comes to mind is that it could take 5 minutes for the updates to the region map for the search manager to take effect (due to caching). Could this be the case?

Thanks,

Scott

Kapil_Chaudhari

I just tried it again. But, I got the same response with no OTModifyTime in it.

Kapil_Chaudhari

Hi Scott,

I got it to work. I also had to mark it as Displayable.

Thanks a lot for all your help.

Kapil

Kapil_Chaudhari

Hi Scott,

Is there a way to query for all the documents which are modified by only querying root folder ?

e.g.- Folder_1 has Folder_2 and Folder_3

Folder_2 has Folder_4 ...Folder_3 has Folder_5.

Can I only query Folder_1 and get all the documents which were modified in Folder_2,Folder_3,Folder_4, Folder_5? I can always provide "AND" separated values of all the folders and get it, but I would like to only provide FOlder_1 nodeid and recurse into the sub-folders.

Thanks,

Kapil

sgrasley

Hi, Kapil.

Your OTLocation query should also return items in subfolders.

Thanks,

Scott

sgrasley

Hi, Kapil.

Having said that, if you're searching inside a project, you might have to include an OTLocation condition for the Project Volume's ID.

Thanks,

Scott

Kapil_Chaudhari

Thanks Scott. It works. But, I found one strange thing. I added a document in one of the folders and sent a weservice request immidiately. I didn't see the newly added document in response. I sent another request after couple of minutes and I got this document in the response.

I was wondering why I was not able to get the document when i sent the first request? Does it take time to synchronize?

sgrasley

Hi, Kapil.

Yes, the search won't return the newly added document until the search engine has had a chance to index it.

Thanks,

Scott

Kapil_Chaudhari

Thanks Scott. Our webservices are deployed on Tomcat 5.5. To call the webservice, we use java based HTTP client and use HTTP POST. Before posting a SOAP envelope, we set the content type to "text/xml".

But, tomcat treats HTTP request and response as ISO-8859-1 encoding type by default instead of UTF-8. Our security token has "%" sign in it and when I use httpClient.post through JAVA, it interprets the % sign wrongly and I get "Couldn't login with cookie" error (error when you provide invalid token).

To get around this problem I set the content type "text/xml; charset=UTF-8" and sent the request. It works fine with this solution. But I would like to change it on server side instead of client side. Do you have any suggestions on how to achieve this on server side instead of doing a code change on client side?

Thanks,

Kapil

sgrasley

Hi, Kapil.

I don't have any ideas. I wouldn't expect the character '%' to cause problems...

Scott

Kapil_Chaudhari

Hi Scott,

How can we mention a date time stamp together to find all the documents added to that folder after that time?

e.g. - I am searching a folder to see what are the new additions to that folder. This is what my query looks like: -

where1=[qlregion OTLocation] 240611&lookfor1=complexquery&boolean2=and&where2=[qlregion OTModifyDate] > 20100927&lookfor2=complexquery&boolean3=and&where3=[qlregion OTModifyTime] > 043557&lookfor3=complexquery&boolean4=and&where4=[qlregion OTSubTypeName] Document&lookfor4=complexquery

Everytime I search the folder, I will save the latest OTModifyTime. Next time when I search the folder again, I will use this OTModifyTime in the query to get all the documents added after that time.

The above query is causing problems because it treats Date and Time separately. Instead of returning all the documents which were modified after 2010-09-27 04:35:57, it only returns documents modified after 04:35:57 on each day and thereby omitting all the documents before 04:34:57. How can we change the query so that it will return everything after that date and time(2010-09-27 04:35:57)?

Thanks,

Kapil

Kyle_Swidrovich

Why not use logic like:

( (Current Modify Date = Original Modify Date) AND (Current Time > Original Time) ) OR ( Current Modify Date > Original Modify Date)

Kapil_Chaudhari

Thanks. I want to see if I can use date and time stamp together instead of coding a logic like you mentioned. If there is no way around, I will have to use that logic.