DD 6.0.2 - filter clarification

I'm reading the documentation, looked at a few DevNet posts, but I'm still a bit confused as to whether I can specify filters in my DD config file to do what I want.

Essentially I'm looking to do two things here:

Globally filter "out" any non-DCRs that might have been passed in through a filelist
Deployment definition level filtering to specify that a given definition applies to only certain DCR types

The first part seems to be something like this: <field name="TeamSite/Templating/DCR/Type" value="[a-zA-z0-9]*"/> however:

It isn't clear if I can reference an EA via this directive if I'm deploying content as opposed to metadata
It's unclear if I can use perlre-type expressions in value - like value="^\w+" so as to avoid having to specify an explicit character-class ([...])
It's unclear if I can use regex-style values or if I have to use SQL-style directives -- the documentation seems to show both the use of things like '*' and the use of '%' within the value-side of a field element

So - given all that confusion - I have to ask:

The second item is within the deployment section, after the source element - where I would place something like: ...
</source>
<filter use="global_filter"/>
...What I'm trying to understand here is whether I can specify multiple filter specifications here, e.g.: ...
</source>
<filter use="global_filter"/>
<filter use="dcr_specific_filter"/>
<filter name="and_this_to">
...
</filter>
...The DTD seems to indicate that this is legal ("(.... filter)*" or "(... | filter | ...)*" - but I didn't see any examples of such in the documentation, so I'm just looking for verification.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Find more posts tagged with

Comments

Adam Stoller

... and while I'm on the subject ....

A clarification on the distinctions / interactions among the filtering directives: keep, discard, and ignore.

If I specify just a keep -- does that mean that anything which does not match is discarded or ignored?
If I specify just a discard - does that mean that anything which does not match is kept or ignored?
If I specify just an ignore - does that mean that anything which does not match is kept or discarded
What is the difference between discard and ignore? - please provide contextual example.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Migrateduser

With using filters, you need to think about the tuple, not so much the DCR. The tuple contains the actual source (EA's + content) that will get deployed. You are filtering basically on the tuple key/values.

What I like to do, is run my deployment and then look at the log file and see what the tuple looks like (you will see a line that says TUPLE={ ....}.
Here I can see what the actual key value pairs are in the tuple and set up the filters based on that.

So you should be able to filter based on EA's as you described as long as both content and EA's are getting deployed together. (For example, use the xml-source element and include-extended-attributes="yes" attribute)

Regarding regex... I'm not a regex person, sorry. But we've generally described it as using perl5 regex. Sounds like you are saying that we acutally use some kind of mix of regex and 'SQL-style'. (Actually, if you can tell me which is which, that would be good for my knowledge.)

Regarding the filter combo... that should work if the DTD says it's suported. I'll try to check it out later to be sure.

Thanks for pointing out these doc issues, points of confusion, etc... we're trying to make the DD docs more digestable. So this helps.

Mariam

PS: The docs for the next service pack(602SP1) of OD clarifies the substitutions/filters section a bit.

Adam Stoller

With using filters, you need to think about the tuple, not so much the DCR. The tuple contains the actual source (EA's + content) that will get deployed. You are filtering basically on the tuple key/values.
[...]
So you should be able to filter based on EA's as you described as long as both content and EA's are getting deployed together. (For example, use the xml-source element and include-extended-attributes="yes" attribute)

Hmm - well, we're not intending to deploy any EAs - just content.
If I added the include-extended-attributes="yes" directive but did not specify any EA-to-column mappings anywhere in the dbschema[.cfg] - would I be able to filter on EAs without having to worry about not deploying them?

Regarding regex... I'm not a regex person, sorry. But we've generally described it as using perl5 regex. Sounds like you are saying that we acutally use some kind of mix of regex and 'SQL-style'. (Actually, if you can tell me which is which, that would be good for my knowledge.)

The perl5 regex's allow things like:
- ^ and $ to anchor a pattern to the beginning / end of a line
- .* as a total wild-card, ?, *, +, {count}, and {min,max} to indicate variations on the repetition of preceding characters and such.
- lot's more.

The SQL syntax (to my limited knowledge) generally uses % as a wild-card - I have no idea if it provides for any of the other directives.

Regarding the filter combo... that should work if the DTD says it's suported. I'll try to check it out later to be sure.

Thanks - it probably does, but I wanted to get confirmation while I'm still in the writing stage before I start beating my head against the wall in the trial-and-error stage ;-)

Thanks for pointing out these doc issues, points of confusion, etc... we're trying to make the DD docs more digestable. So this helps.

Mariam

PS: The docs for the next service pack(602SP1) of OD clarifies the substitutions/filters section a bit.

Glad to help - I may send you a few more things I noticed, under separate cover.

Is 602SP1 out yet? (didn't see it on support site) or is this coming out "soon" ?

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Migrateduser

You don't have to map the EA's in your dbschema... they can be in your tuple, but you don't have to actually deploy them. DD will just ignore that data if it's not mapped to anything.

Mariam

Adam Stoller

Excellent! I'm almost starting to like DD :-)

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Migrateduser

Hmm... we might need you to be our poster child for DD Smiley Happy

Adam Stoller

It depends - does being a poster child pay well ? :-)

Still have unanswered questions in this thread though - so I'll repeat them here for context:

If I specify just a keep -- does that mean that anything which does not match is discarded or ignored?
If I specify just a discard - does that mean that anything which does not match is kept or ignored?
If I specify just an ignore - does that mean that anything which does not match is kept or discarded?
What is the difference between discard and ignore? - please provide contextual example.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com