debugging Search

TS 6.5, SP1 on Solaris 2.8

Configured branch to be indexed

If I search for a filename, nothing was found (file is in Staging) (if I search for a jpg),
if I search for an XML or HTML file, I get the same 2 unwanted files

If I search for the content of an HTML file, it works

If I search for the content of a different file (PDF, TXT, XML) not found.

How can I debug this ?

Tips/Pointers/RTFMs appreciated

Andy

Find more posts tagged with

Comments

skip11

Hi Andy,

I would first (unfortunately) remove the index and and re-index and watch the logs in the
shared file system (where the index is created) for the particular branch where the failed
searchs occur. Before kicking off the indexing, tail -f on sysinfo.log and watch it. If that's
not reasonable (it may take quite a while to completely re-index and people need things like
breaks and lunch and going home ) go back to it later and look for errors where it could not
index a particular file. I've seen them in there, and tested a search on a failed index of a file,
and it was not found. Also, (I'm grasping here) there is a default setting in search.properties
which defines the maximum size that a file can be indexed at 20mb. It's probably not that -
holy smokes a 20mb .html file!, but that may something else to look at. Comments in html
files in the initial part of the file also might fool the index service to think it's not .html - I've
seen that as well. Use command lines tools to ensure the status of the indicies.

iwsrchndxstatus <branch name> is a good one and a status of 11 means indexing is complete.

iwndxlistbr is also one that will show you all branches which have completed indexing.

Hope some of this helps,

R.Barger
Credit Suisse Group
Zurich, Switzerland

nipper

Very strange.

Stopped indexer and search

removed the search db

put 1 branch (only) in my branches.cfg

In that branch I have 5 files:

aussie.txt and aussie.html with johnny and greg

american.txt and american.html with andy adam dave and john

I also dropped a PDF in there.

restart indexer and iw.tssearch:

from index log:

[29 Apr 2005 09:16:50,603] WARN com.interwoven.hunter.index.BranchMapping (main) - Initial database not found. Assuming no branches are indexed.
[29 Apr 2005 09:16:52,418] INFO com.interwoven.hunter.index.IndexServer (main) - Index Server is ready.
[29 Apr 2005 09:16:52,419] INFO com.interwoven.hunter.index.BulkJobProcessor (BulkIndexer-0) - BulkIndexer-0 starting indexing of branch //e08k08/default/main/search_test
[29 Apr 2005 09:16:55,413] INFO com.interwoven.hunter.index.BulkJobProcessor (BulkIndexer-0) - BulkIndexer-0 completed indexing of branch //e08k08/default/main/search_test till edition //e08k08/default/main/search_test/EDITION/#1
[29 Apr 2005 09:16:55,414] INFO com.interwoven.hunter.index.BulkJobProcessor (BulkIndexer-0) - Files processed for branch //e08k08/default/main/search_test is 5
[29 Apr 2005 09:17:24,331] INFO com.interwoven.hunter.index.IndexServer (main) - Accepting connections on port6715
[29 Apr 2005 09:17:24,337] INFO com.interwoven.hunter.index.IndexServer (main) - Index Server is ready.
[29 Apr 2005 09:17:24,375] INFO com.interwoven.hunter.index.IndexManager (Thread-10) - Registered search server host Smiley Tongue

ort = e08k08:6720

so it sees my 5 files.

But I search by file name or by andy, search returns nothing.

nothing in tssearch.log (nothing of note)

bturns

I had similar problems getting the index to update. I noticed that the index doesn't seem to update unless there is a change to a file. If a file has already been submitted then the index doesn't keep reindexing that file. Try the following steps

1) verify that your index was created and updated.
2) make a small change to a file and submit the file. Then manually update the index.
3) verify that the index was updated.
4) perform a search on the file you updated.

1) The index should be located in /data/iwmnt/<index directory> (this is where our is)

<index directory> is a very long directory name with lots of 0's (i.e. 0000000100000000000000510000006400000000000020200000021D0000000000002250_1)

with in this directory navigate to the "parts" folder. In the parts folder you will see the index. See when the last modified date was.

2) To manually update the index run the <iw-home>/bin/iwndxrefreshbr

you may have to stop and start the index and search servers. I'm not sure.

3) return to the index directory and verify that the index was updated. You should see a new index file with today's date and time. This may take a couple minutes. Updates are placed in a queue and processed every couple of minutes.

4) once you've verified that the index was updated. Try and search for the document that you submitted.

Hope this helps. Let me know if you have any questions.

Thanks,

Brian

nipper

added a name

submitted & reindexed

from the log: /usr/search/0000000.../sysinfo.log:

Fri Apr 29 09:16:52 2005
msg(1): Error E0-0613 (Vdb Access): Read out of range (rec=0, numrecs=0)
Fri Apr 29 09:16:52 2005
msg(1): Error E0-0602 (Vdb Access): Error in read_entry: 00000001.ddd, D
AT_OF, 0
Fri Apr 29 09:16:52 2005
msg(1): Error E0-0623 (Vdb Access): Error reading offset of dispatch data
Fri Apr 29 09:16:55 2005
msg(1): Error E0-0613 (Vdb Access): Read out of range (rec=0, numrecs=0)
Fri Apr 29 09:16:55 2005
msg(1): Error E0-0602 (Vdb Access): Error in read_entry: 00000002.ddd, D
AT_OF, 0
Fri Apr 29 09:16:55 2005
msg(1): Error E0-0623 (Vdb Access): Error reading offset of dispatch data
Fri Apr 29 10:12:16 2005
msg(1): Error E0-0613 (Vdb Access): Read out of range (rec=0, numrecs=0)
Fri Apr 29 10:12:16 2005
msg(1): Error E0-0602 (Vdb Access): Error in read_entry: 00000003.ddd, D
AT_OF, 0
Fri Apr 29 10:12:16 2005
msg(1): Error E0-0623 (Vdb Access): Error reading offset of dispatch data

in the parts directory:

-rw-r--r-- 1 root other 31781 Apr 29 09:16 00000001.ddd
-rw-r--r-- 1 root other 57826 Apr 29 09:16 00000001.did
-rw-r--r-- 1 root other 30754 Apr 29 10:12 00000002.ddd
-rw-r--r-- 1 root other 9925 Apr 29 10:12 00000002.did

(10:12 was when I reindexed)

To no avail, still no results in search.

bturns

interesting. Did you try to restart the index server and search server?

nipper

yup. part of the problem is that TS 6.5 SP1 was installed but not TS Search SP1,

testing again, I can search by filename & it works. I can search by content
for some of the files, but not others.

I grepped through the parts directory for the search index and the words
I used were in there.

very strange

Migrateduser

Did anyone who is posting on this thread watch the DevNet webcast on Search Tips we did a while back? I think it might address some of these questions, or at least tell you the way enginering THINKS it should behave.

See:
http://devnet.interwoven.com/site.fcgi/webcasts/docs/webcast-recordings.html#webcast05-15

regards,

lissa