Search incremental indexing

Environment: TS 6.5 SP1 - Solaris

I'm having a problem with search not indexing files as they are submitted.
From looking at documentation and other devnet postings related to this, I've verified the following:

1. In iw.cfg, ew_enable=true

2. We have the event subsystem configured to use an Oracle DB. I don't see any errors in eventsubd.log when the event subsystem is started. Something is updating the database because I have over 190,000 rows in the MESSAGES table and is slowly growing. Not really sure if that is normal/expected. Based on this I am assuming that everything is set up correctly in the jmsconfignew.xml.

3. The following is in the search.properties file:
java.naming.factory.initial=org.exolab.jms.jndi.mipc.IpcJndiInitialContextFactory
java.naming.provider.url=tcp://localhost:3035/
TopicConnectionFactory=JmsTopicConnectionFactory
iw.index.topic.1=Interwoven

I see the submit event in the "iwevents.log", but never see anything in the "iw.tsindex.log" as a result of the submit. If I do a "iwndxrefreshbr" for a branch, it will index the files that have been submitted.

I've restarted/bounced the event subsyste, the index server, search server, and UI webapp (tomcat) and nothing seems to work. Am I missing something or has anyone else had similar issues and if so, how did you get them resolved?

Thanks

Find more posts tagged with

Comments

Migrateduser

If the size of your messages table is constantly growing, I would be suspicious that the events are never being consumed.

jornduff

I was wondering the same thing, that's why I mentioned that the MESSAGES table continues to grow. Can anyone confirm that rows get added to the MESSAGES table when a file is submitted and that the incremental index process should be triggered by these rows showing up in the database? If so, should the rows in the database get removed when the incremental index job is complete. If that is supposed to be the case, I would think that the table would hover around 0 rows when everything is working correctly.

Just a quick update on this. I've contacted support about this and they are looking into it. From what I understand they were able to re-create the same problem.

Frederik

any news on this? Thx.

I'm in a similar situation where I just configured the RDBMS for the event subsystem... there's now 688 rows in MESSAGES table, but the table isn't growing when I do a submit in teamsite. (ts65 sp2 on Sol9)

EDIT:
ok, my dumb fault. I set ew_enable=false while I meant to set [COLOR=Navy]ew_enable=true in iw.cfg. Cfr ts.650a.install.unix.pdf p.65[/COLOR]

Migrateduser

Hi jornduff,
I think you have added the tag for the server for publishing events but not to the indexer for subscribing these events from the event subsystem.

Modify the following tag in the search.properties
# Event-based updates are enabled
iw.index.events.enable=true
After this you must be able to see lot of existing submit events received from the event subsystem.

Also as far as the growing messages table size is concerned, The event subsystem uses this as the persistent storage for the messages yet to be delivered. Once all the consumers (search, reporting, Opendeploy) of the message receive the message the table is cleared of them. Don't worry about this unless the table size grows too much and needs some manual cleanups. This might happen if some component has subscribed and then never comes back.

Migrateduser

Please verify the Search eventsubsystem subscription by looking into the search log files. You should be able to see the Submit events in the log files.
If the eventsubsystem is down the search will not start logging an error.

Frederik

Hi jornduff,

Just to be explicit. I resurrected this thread yesterday because I had a similar looking problem - now solved (see my EDIT in prev post). The original thread is over a year old, and may not need assistance Smiley Wink

vbatsu

Hi,

I have exact same issue now. (TS 6.7.1 on Solaris + Search).
File is lost and never appears again in search once submitted to staging.

Please help.
Thanks

Migrateduser

First check to make sure that your search.properties file has the following as Hemant suggested.

# Event-based updates are enabled
iw.index.events.enable=true

If that is already in there then check the event subsystem log (eventsubd.log) for errors that happen when ESS is running and/or the eventsubd_err.log for errors that occur on startup of ESS. I highly recommend that you use the following KB to add the date stamp to your event subsystem logger so you can tell the difference between old errors and new ones.

https://support.interwoven.com/kb/kb_show_article2.asp?ArticleID=58970

Any errors that event subsystem has will prevent the indexer from firing off at the time of a submit. This will cause the file to "disappear" from the search because it will not have been added to the collection. The file "reappears" when a forced indexing from iwndxrefreshbr or other clt that force a re-indexing of the branch is run.

Fix the errors that you find in event subsystem and make sure (by checking the log) that they are fixed. Then, for pre-6.7.1 SP1 servers, follow this KB for performance tuning your event subsystem:

https://support.interwoven.com/kb/kb_show_article2.asp?ArticleID=59720

After you have guaranteed that event subsystem is running fine, stop and restart the index service. Check the iw.tsindex.log to make sure there are no errors. Finally make a change to a file and submit it to staging (only changed files will log an event). It may take some time for your file to show up but if the indexer consumed the event properly you will see the file in your next search. See the following KB on why files do not appear immediately after submitting:

https://support.interwoven.com/kb/kb_show_article2.asp?ArticleID=58231

If you are still having problems, contact Interwoven support immediately and open a support case for the issue. Provide the following logs and configuration files plus a detail explanation of the issue along with what you have tried.

iw.tsindex.log
iw.tssearch.log
search.properties
eventsubd.log
eventsubd_err.log
jmsconfignew.xml