Can MetaTagger Perform Selective Processing

Hi,
Can we make metatagger do selective processing ?
Suppose I have a scenario where i have only text files to process and all the files are comming from internet.
Now i like to have two content processors A_CP and B_CP.
And In A_CP i like to process X type of text files and
in B_CP i would like to process Y type of text files.
How to do it in MetaTagger where metatagger can recognize files

Find more posts tagged with

Comments

Migrateduser

by, X type and Y type, you mean bascially different file extensions, you can by adding you own filetypeGroup and adding that fileTypeGroup in your content processors:

some thing like:
[your CP config]
A_CP

B_CP

[your fileTypeGroupConfig]
XType

YType

jonak

Hi snkmoorthy,
Thank you for your reply.
But X type or Y type are not file extension.
X type files may be book type
Y Type files may be News letters type
But both file types are in text format

Migrateduser

What is your source for the 'book' and 'new letter' types?

jonak

The Source for the document is Internet. It means that these files are downloaded from the web

Migrateduser

ok,got it!

1. Books are named - book1.txt,book2.txt or rand2.txt
2. Newsletters are named - newsletter1.txt,newsletter2.txt or some rand1.txt

if you are asking MT to use different taxonomies(projects) for (1) and (2), it is not possible, because we have not made any distinction for the content type(read, file type base on extension, MT will run the text CP if it gets such ".txt" files)

May I ask, why you do not want the Books to run through B_CP and Newsletters to run throught A_CP content processors or why you want to run them in different CPs?

jonak

I have Different Set of Meta tags for Book type
Suppose Meta tags are Book Name Author, Publisher, Edition.
Suppose Meta tags for News letters are Date, Subject, From whom etc

If i have different types of processors for Book and Newsletter. Book content processor should process only books and
News letter Content processor should process only news letters.

Migrateduser

Since our tags are going to come from the content itself rather than a taxonomy, I guess you will be writing extractors and some pre-processors to generate the tags, something like pattern1 - tag1
pattern2 - tag2

if we add such projects, we get into a situation where, MT will run the extractor and pre-processors for both Books and Newsletters. If the tags generated are mutually exclusive, that is, Author tag can come only from Books and "From whom" is named NOT as Author but as "From" for Newsletters, let MT run, you can figure out which came from which by looking at the tags(not a path breaking solution, I know)

Another not so path breaking solution is, if we know we are tagging Newsletters only and book only in seperate time, you can modify the extension of the newsletters(temporarily) to .nlt have a ntl fileTypeGroup and add your news letter specific extractors/pre-processors in to a NLT CP, capture the tags and switch the extension back to .txt

Migrateduser

One additional tool is the regex filter for each content processor. It will allow you to filter what the CP processes by matching the submitted file path against the regular expression.

Migrateduser

One additional tool is the regex filter for each content processor. It will allow you to filter what the CP processes by matching the submitted file path against the regular expression.

Ok, this is wonderful
With cbreyman's info, we can do the following:

txt

A_CP
text
project1
Books/*

B_CP
text
project2
News/*

A_CP will run only if the file path has "Books" in it, and B_CP will not run on it and if there is News in the file path, A_CP will not run, but B_CP will run

not need to change the file extension - this is NEAT!, however if there is no clear seperation in the file path, and it can change or there are many you should add them with more elements

jonak

Sorry i could not catch the discussion at proper time but i have a small issue.

My documents are coming from Internet from scientific sites. I can get news letters from those sites while downloading or I can get books while downloading. We are not storing the files before tagging so we might not be in position to store the files in a book folder or newsletter folder.
and getting path from the net is some times difficult.

Migrateduser

How about a post processor? Create a Classified and train it with Book files and News files then ask it to classify each file as it comes through as either Book or News. Then use a post processor to remove fields and values you do not need in the metadata record depending on the value of the Classifier field.