nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Avoiding large files

System

Some of our users are putting huge pdfs into the system. While we generally want metatagger to tag PDFs, we don't want it to do so on files that are going to take an inordinate amount of time to process. Is there any way to configure a conditional limit on a filetype so that if it is larger than a given size, it will just return empty metadata?

Find more posts tagged with

Comments

Migrateduser

The folks I checked with suggested that the best way to do this is to have TS workflow skip the set metadata step if the attached file is a big pdf.

hth,

lissa

nipper

We went through that, tagged everything initially via script & then had another script to preload the required MD on any new objects (when
imported > 10 GB).

Andy

Migrateduser

That's unworkable for us in the workflow architecture we currently have (which is basically a few modifications I've made to the IWOV PSO deliverable). Because all the metadata is data-deployed into a table which will not accept nulls, some data has to be there. I'm partial to Andy's idea of writing a script to pre-tag things.

nipper

FYI,

The EAs set a re not super well documented. It took a while to reverse engineer how MT sets the EAs and replicate. There is an
EA that acts as collection of all & that can span 2 EAs when you set it via commandline. Try setting some and then using Set Metadata
to open the MD window. When the data in that window is perfect, you are done. That took us the longest time.

Andy

Migrateduser

You might still be able to use workflow for sorting out the huge files. In the initiator staging, check out all the files. For all the big files, remember them by put them in certain job variables.

Right after the CGI task for tagging, you can use an externaltask to set EAs for those files and then attach them to the workflow. This way, you do not have to change your workflows too much and also achieve your target.

Ottawa_IWOV

i am trying to pass a PDF to MetaTagger using MetaTagger.pm
HTML docs are fine, but I get XML parser errors when I pass a PDF. Anyone know if there are 'known' limitations w/ this module?

Migrateduser

An additional approach to large files is to add a CLT preprocessor to MetaTagger that checks the size of the crackedText and truncates after 100K or so.
MT 3.6 has lower size limits than 4.0.1.

One test you can run is to try iwmttransconverter on the offending file to see if it's a problem with conversion to text or the actual text processing size.

What version are you running?

Clark