nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Metatagger: ERROR: File is bigger than maximum siz

Michael

Hi

We are using MetaTagger 3.5.0 on Windows running with TeamSite 5.5.2 on Unix. After importing a large PDF file (>4Mb) metatagger gives the following error after selecting File -> Set Metadata.

ERROR: File is bigger than maximum size allowed

If the file is of type .zip then the error is not received.

I have checked the MetaTagger documentation and I couldn't see anything about a maximum file size or anywhere where this could be configured.

Does anyone know if this can be configured? If not, does anyone know where it is documented what the limit is?

Thanks in advance.

Cheers
Michael

Find more posts tagged with

Comments

Adam Stoller

I believe that there is currently a maximum of 2MB for PDF files (someone else will hopefully verify or correct me here).

I don't think you'll run into the problem with the zip file because the zip file is probably (again, subject to correction) not being parsed. The PDF is being parsed to by the various preprocessors you've associated with PDFs.

--fish
(Interwoven Senior Technical Consultant)

Michael

Thanks for the reply.

I agree that it is only for files which are parsed. Example: I can reproduce the error with files of extension .doc also.

Do you know where that maximum of 2MB is documented?
Do you know if it is configurable?

Thanks for your help.

Cheers
Michael

Adam Stoller

I'll have to defer to someone else who's more experienced/knowledgeable regarding MetaTagger. Sorry.

--fish
(Interwoven Senior Technical Consultant)

Michael

This is what we have received from Interwoven support:

You will need to increase the threshold size for the max allowable size, but
you should do so with caution since this increases the memory requirements.
Please add the 'thresholds' attribute to the metatagger.cfg file:

<thresholds>
<post-crack-threshold>numberOfBytes</post-crack-threshold>
<total-file-size>numberOfBytes</total-file-size>
</thresholds>

The post-crack-threshold default is 1000000 (1 million bytes); this is max
size of plain text after cracking. The total-file-size default is 5000000 (5
million bytes); this is the max file size. Increasing these values uses more
memory and should only be done with caution.

Your particular problem relates to total-file-size attribute and is the max
file size of the file submitted to the MetaTagger server.
Post-crack-threshold is the max size of the plain text that the MetaTagger
server processes. The distinction of the two thresholds is relevant when a
<converter> (e.g. word doc to plain text) is invoked. E.g. a 4MB word doc
might only have 4KB of plain text.

If you know you have large binary files that crack down to relatively small
plain text size, then you may want to increase total-file-size. Increasing
post-crack-threshold is potentially dangerous, however. In increasing
post-crack-threshold, the memory use can increase much more rapidly than
linearly, and processing a large blob of plain text could bog down server.

So you may want to consider an increase in total-file-size if you know you
have large files that crack down to small text size. If necessary, increase
post-crack-threshold incrementally with caution.