nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Dev Server - Unable to index branches in 6.7.1

nipper

So I am using Win2K dev server with search on the same box as TS. With 6.5 (even 6.7) I was able to index the branches. But now I can only index the small branches. The large ones (tens of thousands of elements) seem to hang the CPU. Basically the box is at 100 % CPU utilization (java process) , then IWSERVER and the Y drive become unresponsive. It stays in this state until I bring down the indexer.

Has anyone seen this or have any ideas ?

Andy

Find more posts tagged with

Comments

gzevin

So I am using Win2K dev server with search on the same box as TS. With 6.5 (even 6.7) I was able to index the branches. But now I can only index the small branches. The large ones (tens of thousands of elements) seem to hang the CPU. Basically the box is at 100 % CPU utilization (java process) , then IWSERVER and the Y drive become unresponsive. It stays in this state until I bring down the indexer.

Has anyone seen this or have any ideas ?

Andy

Andy,

is the serch part running on the same box? AFAIK for large implementations it's advisable to keep the search on a separate box. this is what I will be doing here soon.

nipper

Yea but for a dev server, that is not heavily utilized, one would think that it could handle search. Once the branches are indexed, it certainly will. But in the mean time it is hosed. If there was a way to throttle back the indexer, let it work slowly and not use 100% of the CPU, I am certain the system can handle the job. It worked fine (albeit slow) in 6.5 & 6.7

gzevin

Yea but for a dev server, that is not heavily utilized, one would think that it could handle search. Once the branches are indexed, it certainly will. But in the mean time it is hosed. If there was a way to throttle back the indexer, let it work slowly and not use 100% of the CPU, I am certain the system can handle the job. It worked fine (albeit slow) in 6.5 & 6.7

hm. sounds strange. but somehow I'm not surprised Smiley Wink

Migrateduser

The Search is strongly recommended to be installed on the separate machine especially when the branches are huge. The reason for your server being non-responsive most likely is that the initial bootstrap indexing for the branch is a high CPU/memory task as the indexer has to fetch/extract/index every document in that particular branch. One thing you can try is run the indexer when the load is low and you can stop the other non-required processes like Search(other search service than indexer), ServletD, EventSubsystem, Web Demon etc. and then try to get the branches indexed one by one using the CLT's. Once the branches are indexed u can make other services up as now there will not be much load for the indexer except for the index incrementation.

Migrateduser

Yea but for a dev server, that is not heavily utilized, one would think that it could handle search. Once the branches are indexed, it certainly will. But in the mean time it is hosed. If there was a way to throttle back the indexer, let it work slowly and not use 100% of the CPU, I am certain the system can handle the job. It worked fine (albeit slow) in 6.5 & 6.7

Also to decrese the indexer CPU utilization u can decrese the number of the index agents in search.private.properties and then try indexing.
######################################################################
# Index agent configuration
######################################################################

# Number of index agents
iw.index.agent.count=4

# Note: the sum of the number of bulk and incremental threads
# must always be one less than the agent count.
iw.index.bulk.threads=1
iw.index.incremental.threads=1

nipper

Cool, I was digging through the files hoping there was a way to throttle it back, you obviously have found it, but correct me if I am wrong, those look pretty throttled back already.

Is there a way to turn off full text search ? I would be happy with only file name and EA.

Andy

Frederik

Is there a way to turn off full text search ? I would be happy with only file name and EA.

perhaps you could do so by adding more extensions to the list :

# Binary extensions. The content in these files will not be indexed, although metadata will be.
iw.index.binaryextensions=exe,jpg,gif,psd,tif,tiff,au,wav,bmp,rif,zip

not sure how you would go about adding files that have no extensions (like DCRs typically)...

This is the kind of setting we would probably want to set on a per branch basis, though. (found the above in ts 6.5 search.properties; dunno if 6.7 can be configured more selectively)

nipper

perhaps you could do so by adding more extensions to the list :

# Binary extensions. The content in these files will not be indexed, although metadata will be.
iw.index.binaryextensions=exe,jpg,gif,psd,tif,tiff,au,wav,bmp,rif,zip

not sure how you would go about adding files that have no extensions (like DCRs typically)...

This is the kind of setting we would probably want to set on a per branch basis, though. (found the above in ts 6.5 search.properties; dunno if 6.7 can be configured more selectively)

Good thought, all my DCRS are .xml so I can add HTML & XML to that list & trick it.

Andy