Has Document been indexed or not

Does anyone know if there is a way of identifying documents or items that have not been indexed ?

For example if I had an object ID is there a way (other than looking at DTreeNotify to see what is left to process) of seeing that the object has been indexed ?

I am aware of the index verification report (?func=index.verifyagentadmin) but would like to know if a table or tables could be queried that could help to get this information on an individual object basis ?

Any help much appreciated.

Steven

Find more posts tagged with

Comments

appuGmail

The search process is an API to the indexing mechanism as such Oscript or database queries are used to issue search queries etc . If you repost this in the search and indexing forum then they will give you command line help to use the OT search.jar to do diagnostics

As such any object is accessible by search only if it goes thru extraction-> DCS-> UD

The search grid documentation explains all this myriad mechanisms very clearly

Ciao, Appu

On Nov 11, 2016, at 4:12 PM, eLink Entry: Content Server LiveReports Forum <livereportsdiscussion@elinkkc.opentext.com> wrote:

eLink : Has Document been indexed or not
Has Document been indexed or not Posted byHerring, StevenOn 11/11/2016 07:10 AM

Does anyone know if there is a way of identifying documents or items that have not been indexed ? For example if I had an object ID is there a way (other than looking at DTreeNotify to see what is left to process) of seeing that the object has been indexed ? I am aware of the index verification report (?func=index.verifyagentadmin) but would like to know if a table or tables could be queried that could help to get this information on an individual object basis ?Any help much appreciated. Steven
[To post a comment, use the normal reply function]
Forum: Content Server LiveReports Forum
Content Server: Knowledge Center CS16

Kyle_Swidrovich

Steven,

If it’s indexed, then it should be searchable.

The search index is not stored in the database nor is an indicator as to whether or not the search index has an entry for the object.

Just search for “OTDataID”:2000 as an example to see how to return the Enterprise Workspace.

You can change that ID to test any other object, given that you have sufficient permissions to see the object.

Kyle

Steven Herring

Kyle (and Appu),

Thank you for the replies and sorry for the slow response.

My issue (which perhaps I should have included in my original post) is that I noted that a number of (in my case XML documents) were not text searchable - meaning that I could not find them using text contained withni the document. I downloaded the respective documents and added them to a dev box where I have full access to the logs etc and discovered that the documents that I could not find were indeed omitted from the full text index - falling foul of the bad object heuristic (the general idea that 'bad' documents contain lots of unique words).

It seems that using your query Kyle only proves that the document has been processed (and most likel;y the metadata has been extracted and indexed) but in fact using the following search (CS10+) seems to get me those documents that have been processed but marked as not suitable for indexing (although it also includes any infamous zero byte documents):

OTCheckSum:0

(Described in the document 'Open Text Content Server 10.0.0 - Advanced Indexing and Searching Guide English (LLESSRC100000-GGD-EN-1).pdf' as 'allows searching for all documents whose content is empty or did not have their content indexed. Such objects have a content check sum value of 0')

Although it is probably now not the right forum to ask the question having identified these unindexed documents leads me to wonder about those XML documents that have only been partially indexed - a real problem for me and a project I am working on as the belief was that any and all of the content in the XML files was completely indexed and thus searchable........:) ?

Steven

Patrick Pidduck

Hi Steven,

This one isn't simple to answer, and unfortunately like many topics the answer tends to drift and become incrementally better with newer updates.

When objects are indexed, there are a number of indicators that are associated with the object that can provide clues as to how well it was indexed. The checksum is one. Another is the OTContentStatus region, which may contain codes that indicate whether various problems were encountered (unreadable data, truncation, bad heuristics, etc.). I would recommend checking out the 'reserved regions' section of the "Understanding OpenText Search Engine" white paper in the Champion Toolkit for more information.

Patrick