Index phrasing

Can anyone help me out with trying to create a phrase index for a summary.

I have set up the following file:

phrasetrain.cfg

<index>
<lexicon>notes</lexicon>
<script>tokenize -html; splitter; index -create; index_phrases -create -ignorepunctuation -ignorenumbers -ignoretags</script>
<extension>xml</extension>
<path>$DB_HOME/contents</path>
</index>
~
and

phraseindex.cfg

<index>
<lexicon>notes</lexicon>
<script>tokenize -html; splitter; phrase; tagger; index -create -indexphrases</script>
<extension>xml</extension>
<path>$DB_HOME/contents</path>
</index>

when I run
iwgenindex -config phrasetrain.cfg -verbose

I get:
Extract candidate phrases...
Sorting ...
Sorting ...
Compute global term weights...
Compute local term weights...
Compute document vectors...
Compute document lengths...
Commit changes.
ERROR: lexicon is empty

when I run
iwgenindex -config phraseindex.cfg -verbose

I get:
ERROR: Failed to open index 'notes_phrases': Db: Smiley Surprised

pen: No such file or direct
ory
Usage: iwgenindex -config fn.cfg [f1...fn] [-help] [-noconvert] [-v] [-verbose]

My file type for xml is set up in metatagger.cfg as follows:

<fileType>
<label>XML Document</label>
<extension>xml</extension>
<converter>iwhtml2txt -all -converter "iwmtconverter -metadata "
quot;%FILE%"</converter>
<preScanner>HTMLGeneric.ipr</preScanner> <preprocessor>discoverPreProcessor.ipr</preprocessor>
<htmlMode>false</htmlMode>
</fileType>

Any ideas would be greatly appreciated
thanks
cjk

Find more posts tagged with

Comments

Migrateduser

In phrasetrain.cfg add the -create option to tokenize:
<script>tokenize -create -html; etc. etc.
This is a new lexicon, so we have to tell Metatagger to create.

In phraseindex.cfg add the -addterms option to tokenize:
<script>tokenize -addterms -html;
In this case we just want add new terms to the previously created lexicon.

DATA1_INV_NR.pdf

cjk

Thanks jpierre I did that but alas exactly the same errors!

Adam Stoller

Did you stop and restart the MetaTagger service(s) after making the change?

--fish
(Interwoven Senior Technical Consultant)

cjk

yes.

Migrateduser

I didn't look at your original message carefully enough. From the output of the tool it looks like it's not indexing any docs. Do what I suggest before, but also try this:
in the <path> tag use the fill system path not the $DB_HOME environment variable.

Be careful about using iwhtml2text and HTMLGeneric preprocessor on XML...it's not quite the same thing as HTML. Reading XML with an HTML parser will work most of the time but could lead to unexpected results in some special cases.