tsindex errors

TS 6.5 SP1
Solaris 9

I'm seeing the following messages in the iw.tsindex.log after running: sudo /etc/init.d/iw.tsindex start:

 
...
INFO com.interwoven.hunter.index.IndexManager (Thread-0) - Registered search server hostort = wdchqdtmsp01:6720
WARN com.interwoven.hunter.index.BranchMapping (Thread-0) - Unable to get information from TeamSite about branch //wdchqdtmsp01/default/main/ED/en/OTHER. It will not be available for indexing.
[...]
WARN com.interwoven.hunter.index.BranchMapping (Thread-0) - Unable to get information from TeamSite about branch //wdchqdtmsp01/default/main/IWHOME. It will not be available for indexing.
INFO com.interwoven.hunter.common.agent.AgentService (Thread-0) - Waiting for agent service to be initialized
INFO com.interwoven.hunter.filter.DocumentFilter (BulkIndexer-0) - KeyViewFilter error  when filtering the document: /legislation/FedRegister/announcements/1997-2/051697b.pdf
ERROR com.interwoven.hunter.index.IndexerSupport (BulkIndexer-0) - Filter error processing document /legislation/FedRegister/announcements/1997-2/051697b.pdf in branch //wdchqdtmsp01/default/main/ED/en/OTHER
INFO com.interwoven.hunter.filter.DocumentFilter (BulkIndexer-0) - KeyViewFilter error  when filtering the document: /legislation/FedRegister/announcements/1997-3/082297a.pdf
ERROR com.interwoven.hunter.index.IndexerSupport (BulkIndexer-0) - Filter error processing document /legislation/FedRegister/announcements/1997-3/082297a.pdf in branch //wdchqdtmsp01/default/main/ED/en/OTHER
INFO com.interwoven.hunter.filter.DocumentFilter (BulkIndexer-0) - KeyViewFilter error  when filtering the document: /legislation/FedRegister/announcements/1997-4/112497d.pdf
ERROR com.interwoven.hunter.index.IndexerSupport (BulkIndexer-0) - Filter error processing document /legislation/FedRegister/announcements/1997-4/112497d.pdf in branch //wdchqdtmsp01/default/main/ED/en/OTHER
...

For the warnings - if I can do the following:[wdchqdtmsp01] ~ > ls /default/main/ED/en/OTHER
/default/main/ED/en/OTHER:
EDITION/ STAGING/ WORKAREA/
[wdchqdtmsp01] ~ > ls /default/main/IWHOME
/default/main/IWHOME:
EDITION/ STAGING/ WORKAREA/ Why is the indexer "unable to get information" about these branches?

When there is a KeyViewFilterError - is there anyway to get more details about what the error is?

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Find more posts tagged with

Comments

Adam Stoller

Apparently there is an SP1 for Search as well as for TS - I'm downloading it now and hoping it will resolve these problems ...

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Ottawa_IWOV

I had problems with SP1 for Search. Got a bunch of stack trace errors upon installations. Manual says nothing "preparation" wise is necessary. Just double-click and play mokey.

Let me know if you get a successful installation of Search SP1

Thanks,

Migrateduser

As far as why you get error messages while filtering a document, these are errors that are returned back from the backend Verity filter when it tries to crack the document so that it can index the content inside the document. When one gets errors of this type, typically the metadata associated with the document will be indexed and hence it will be available for searching. However, the content inside the document will not be indexed and hence it will not be available for full-text search. Suffice it to say that this is probably a limitation of the search engine that we use in the back-end and it will go away as the search engine is beefed up in the future. In our experiments, we noticed that this kind of problem happens in a very small percentage of documents in a typical content repository.

I am not sure about the problems related to installation of SP1 or why branch information is not obtained by the index server from the relevant TeamSite server.

Regards
--Anirudha

Johnny

Where can I download the search SP?..

The teamsite 6.5 patch area has one download for teamsite and I didnt see anything in the dropdown mentioning search.

John Cuiuli

nipper

I assume that was TS 6.5 SP1, check the readme, there were some changes for tsindex

me

Migrateduser

Hope that you have find it by now. If not yet, you can find it on the product download page under TeamSite 6.5. There both SP1 for TS6.5 and SP1 for Search are available. It's confusing to me, too, why it's not available under the patch download page.

Johnny

Thanks!.. yeah that's a little strange

John Cuiuli

After a reboot - the index server seems to have managed to come up - but I'm still seeing messages like:

[01 Apr 2005 15:17:07,634] WARN com.interwoven.hunter.index.IndexerSupport (BulkIndexer-0) - Date  not in correct format, or format specified (MM/dd/yyyy ) is invalid.

which bothers me - for one thing - should I be concerned about the fact that there is a space before the close-paren in that message - there isn't a space in the config file. The syntax looks correct.
Of course it would be nicer if I knew what file was being looked at (some of the messages tell you that - this one doesn't)
It would be nicer still if I could tell what branch was being processed (none of the messages seem to include that information)

I tried stopping and restarting the indexer with /etc/init.d/iw.tsindex stop and /etc/init.d/iw.tsindex start - but it complains about ports being in use (6716 and 6717) and fails to start then.

I checked with netstat and see:[NEWPROD] ~ > netstat -na | egrep "671[67]"
*.6717 *.* 0 0 49152 0 LISTEN
*.6716 *.* 0 0 49152 0 LISTEN
127.0.0.1.32953 127.0.0.1.6717 49152 200 49152 0 ESTABLISHED
127.0.0.1.6717 127.0.0.1.32953 49152 0 49152 201 ESTABLISHED
127.0.0.1.32954 127.0.0.1.6716 49152 0 49152 0 ESTABLISHED
127.0.0.1.6716 127.0.0.1.32954 49152 0 49152 0 ESTABLISHED
127.0.0.1.32955 127.0.0.1.6717 49152 0 49152 0 ESTABLISHED
127.0.0.1.6717 127.0.0.1.32955 49152 0 49152 0 ESTABLISHED
127.0.0.1.32956 127.0.0.1.6716 48991 0 49152 0 ESTABLISHED
127.0.0.1.6716 127.0.0.1.32956 49152 0 49152 0 ESTABLISHED
127.0.0.1.32957 127.0.0.1.6717 49152 0 49152 0 ESTABLISHED
127.0.0.1.6717 127.0.0.1.32957 49152 0 49152 0 ESTABLISHED
127.0.0.1.32958 127.0.0.1.6716 49152 0 49152 0 ESTABLISHED
127.0.0.1.6716 127.0.0.1.32958 49152 0 49152 0 ESTABLISHED I found the process that's holding it open:

   733 ?        R  7:44 /usr/app/iwov/search/jre/bin/java -server -Xms64m -Xmx512m -Diw.search.base.dir=/usr/app/iwov/search -Djava.library.path=/usr/app/iwov/search/lib:/usr/app/iwov/search/verity/filter/lib/sol -jar /usr/app/iwov/search/lib/hunter_index.jar

I've already run /etc/init.d/iw.tssearch stop and /etc/init.d/iw.tsindex stop - so what is this process and why is it still there?

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Johnny

I've noticed processes not stopping properly with search.

I keep a pause between the stops / starts.. and then kill the left overs between.

Put up the logging to DEBUG in SEARCH/etc/search.properties

log4j.rootLogger=DEBUG, stdout

You might get more info on what's going on.

John Cuiuli

Migrateduser

I also got a lot of error like this one:
Date not in correct format, or format specified (MM/dd/yyyy ) is invalid.
Before and after installation of SP1. I'm assuming that it is cause by the date field EA on my content files since we have a optional date field, which can be blank if our users don't set it. I'm assuming it's annoying but harmless. Am I wrong?

Migrateduser

I've sent this information on to the SP1 team. I'll let you know when I learn more.

lissa

Migrateduser

OK, please note the KB up on the support site about a problem with the installation of search for 6.5 SP1:
https://support.interwoven.com/kb/kb_show_article2.asp?ArticleID=54052

regards,

lissa

skip11

Hi,

I believe this is a bug, and have filed a case against it. The resolution I have seen (Solaris 8) is to wait
approximately 2 minutes after stopping the search server. When your ports are no longer in TIME_WAIT
in netstat -na, stop the index server and most times all is well. I was told this bug was addressed last
August and should have been fixed inthe initial release of the product. Looks like a regression to me.
I have reproduced the problem many times and will likely reopen the case again.
Case 1215635, Credit Suisse, Switzerland.

Regards,

R.Barger
Credit Suisse Group
Zurich, Switzerland

Adam Stoller

I ended up doing a kill -9 of the process - and eventually managed to get all of our branches indexed (or so it seems - planning to check things out today -- but I got a LOT of warning messages in the log files ....)

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

skip11

Hi,

It's clear to me that the ports are not being shutdown gracefully. Running
truss -p on the java processes associated with index and search and then
shutting the app down, shows the ports are not being closed in a timely fashion.
Likely they are waiting for some interprocess comunication to resolve. The
problem is excerbated when running the search services on a remote system from TeamSite.

Regards,

R.Barger
Credit Suisse Group
Zurich, Switzerland