nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Deconstructing a DCR?

Adam Stoller

Environments[indent]TS 672 (unclear of service packs and patches)
TS 7.2
W2K3[/indent]Task[indent]I want to take a DCR ("xml" type in this case) and "deconstruct" it into a series of FormAPI calls. Yeah, it sounds weird, but I'll try to explain...

We have DCTs that contain collections of fields for different language / locales and when the DCR gets saved, we want to split it into distinct DCRs for each language / locale type (each "component" DCR goes into it's own branch's templatedata/category/data/ directory with the same name, and will have the proper EAs on them) - although there are some tricky issues with this aspect, that's not the focus of this task.

When you edit one of these "component" DCRs (in any of the language / locale branches) - the idea is to merge all the content from all the associated language / locale component DCRs into the form so that you once again see the "full" DCR. (not my desired architecture, but it's what I've got handed to me).

Thus far, the process has been to slurp in the DCR data from the component DCRs and merge them at the file system level and then force (or provide a mans to force) a refresh of the Edit DCR window so that all the data shows up. This has a number of problems which I won't go into right now.

The idea I'm thinking about is to take each of the [other] language / locale DCRs and, using a callserver script, deconstruct the DCR into a series of xpaths from which I can then make FormAPI calls to populate their data into the open form - without modifying the actual DCR at the file system level.
Theoretically, this doesn't sound too bad.

Unfortunately, I'm having a bear of a time trying to figure out how to get the proper xpaths for each item I find in the DCR.[/indent]Approaches[indent]My initial approach was to use something like TeamSite::XMLnode to parse the DCR and recurse through it via things like get_node_list(). This works up to a point.
I can get all the individual element names, I can even [right now] get someone accurate xpaths (I need to play around with stacks to get the right push / pop algorithm) - but what I cannot figure out is how to determine:

Find more posts tagged with

Comments

oduiubo

Well, base on your requirement, if I understand correctly – you have to have master DCR which will be dynamically constructed or assembled from several others sources/DCRs or will create master record and list of children which can be editable by it self as well.
If I understood correctly, it will be some complex solution for this requirement.
Master DCR must hold links to other children DCRs. Use Call Server or AJAX for save children purpose service. Then need to have assembling on load mode service which could be AJAX base or inline substitution method. That's it.

I am using similar logic from 2007 for full DCR language localization which required for and by Canadian Government. I am using XML and Perl only for that solution which has been implemented at several Government projects.

ISCBorisB

I am using similar logic from 2007...

No you don't. Fish builds a collection of "member" DCRs Items' XPath, not just a list of children DCRs.

Thus the main issue is: how to determine without parsing DCT if any Container/Item in the XML DCR(s)
is a replicant? Have any reasonable ideas about this?

Adam Stoller

Boris is correct that oduiubo is incorrect ;-)

FWIW - I think I got pretty close on Friday. Instead of parsing the DCT (which is difficult to say the least because the DCT is generated on-the-fly via inline calls), I opted to require the creation of a simple text file (dctinfo.txt) that sits in the same directory as the datacapture.cfg file and lists the xpaths for the replicants - something like:

# This file lists replicant xpaths (without replicant notation)
# All blank lines and those beginning with '#' are ignored
/root/non_rep_container/rep_container
/root/rep_item

I've gotten to the point where my code correctly identifies the items within a replicating container but doesn't quite get the replicant notation correctly set on the xpath for those items, and I still need to adjust the code to identify replicating items. I think I can get this working before EOD Monday (hopefully).
With regard to the above dctinfo.txt file - the code assumes that if the file is not there, or does not contain any non-commented-out entries, that there simply are no replicants in the DCT.

The good news is that, without replicants, the code seems to work well, updating the form with the information from the [other] locale-specific DCRs without requiring a refresh, and thus far (in very simple test-case scenarios) seems to provide reasonable performance.

Once I get the replicant piece working - we'll have to build out more complex test cases (lots of other locale-specific DCRs with lots of fields, etc.).

Anyway, if anyone else has reasonable input to provide, I'm interested in hearing it - but please - make sure you're responding to the issue at hand and not just dumping irrelevant ideas into the thread.

nipper

One tangential comment I can provide is that when I did something similar, instead of using the .txt file, I used an Admin DCR. Really does not make a significant difference but the utilities for parsing XML in Perl are quite useful.

Adam Stoller

One tangential comment I can provide is that when I did something similar, instead of using the .txt file, I used an Admin DCR. Really does not make a significant difference but the utilities for parsing XML in Perl are quite useful.

When you say "similar" - how similar?
Did you have to properly determine the xpaths for replicant containers and items? If so, did you also handle nested replicant containers?
Care to share any pointers in that regard (here or via private mail)?

Adam Stoller

Okay - the code I've written appears to work sufficiently for non-nested replicants which is apparently all we need to support right now (I'm not "happy" about not getting it to work for nested replicants, but I'm willing to let that go for now).

I'm now running into the dreaded "character-encoding" (or similar) issue - where I'm reading a bunch of stuff from a pre-existing [locale-specific] DCR and attempting to turn it into a FormAPI directive within Javascript being printed to STDOUT (as part of a callServer script) in order to get those values populated into the current Edit DCR form.
The basic reading code is:

if (open(IN, '<:encoding(UTF-8)', $dcr) { ... }

If I just print the data back out to the browser via STDOUT (

print $html

) then I see something like the following:

â€¢ Jusquâ€™Ã  sept grammes de protÃ©ines de soja et de fibres saines.
â€¢ Comprend des vitamines C et E.
â€¢ Se mÃ©lange avec des Ã©pices pour crÃ©er vos propres recettes.
â€¢ En poudre.

If I replace those **** Microsoft bullets with:

($html = encode_entities($html)) =~ s|\&bull;|&#8226;|gsm;
print decode_entities($html);

I see something like this (the bullets look more correct on-site than they do in the forum posting):

• Jusquâ€™à sept grammes de protéines de soja et de fibres saines.
• Comprend des vitamines C et E.
• Se mélange avec des épices pour créer vos propres recettes.
• En poudre.

It looks much better, but there still seems to be some characters / character-sequences that aren't getting translated correctly.

Note: I tried the above code both with, and without:

for)

ISCBorisB

Properly formatted DCR already has UTF-8 encoded elements. So, in theory neither STDOUT Filehandle binmode
nor "from_to" Encoding should be your concern. As long as you obtain DCR's data as provided (eg, in UTF-8)
you should be fine.

What you should do is indicate callServer's CGI-output encoding, either through "Content-Type" http Reply Header
or by formatting corresponding HTML Meta. I've included a (hopefully) working sample, it actually uses the latter
method to set some Russian text in the DCR.

Couple caveats:
- MS bullets are actually secondary for this discussion. If I were you I would simply double check that I still
have correct UTF-8 after substitution.

- My apologies if I state the obvious, but just in case...Multi-line Values can easily cause JS Syntax errors,
like for example Multi-line value in my api.set_text() sample. Replace \n with \\n or use multiple call-backs
inserting EOLs on JS side, etc

P.S. I do not *believe* that you can reliably build an XPath for nested replicants without parsing DCTs - directly
or through some undocumented API. The information is simply "not there".

Pie Chart.rptdesign

Adam Stoller

I was hoping you'd respond, since I know this kind of thing is, shall we say, one of your strong suits.

Properly formatted DCR already has UTF-8 encoded elements. So, in theory neither STDOUT Filehandle binmode nor "from_to" Encoding should be your concern. As long as you obtain DCR's data as provided (eg, in UTF-8) you should be fine.

What you should do is indicate callServer's CGI-output encoding, either through "Content-Type" http Reply Header or by formatting corresponding HTML Meta. I've included a (hopefully) working sample, it actually uses the latter method to set some Russian text in the DCR.

I hadn't thought of that - I'll definitely check into that when I get to the office. Thanks!

Couple caveats:
- MS bullets are actually secondary for this discussion. If I were you I would simply double check that I still have correct UTF-8 after substitution.

I agree, I think they actually came along okay to begin with - the problem I ran into originally (that made me put in the code for the bullets) was exactly what you bring up below...

In the meantime - if the encoding on the output from the CGI doesn't fix the current problem - which I believe is an apostrophe, I may still need to do some character replacements.

- My apologies if I state the obvious, but just in case...Multi-line Values can easily cause JS Syntax errors, like for example Multi-line value in my api.set_text() sample. Replace \n with \\n or use multiple call-backs inserting EOLs on JS side, etc

Heh, after putting in the code for the bullets and still getting a JS error - I looked closer at the error message and realized that there was a multi-line problem. I replaced the newlines with line breaks (the text value here is going into a TinyMCE window) - though I thought about it as I was getting ready to leave the office yesterday and though about just escaping (\\\n) the newlines - I may change that too.

P.S. I do not *believe* that you can reliably build an XPath for nested replicants without parsing DCTs - directly or through some undocumented API. The information is simply "not there".

Well - for single-level replicants I was able to do it by just listing the non-replicant-annotated replicant xpaths in a simple text file and using them (in reverse-sorted order) to process the xpaths I had found, but as you say, there's nothing OOTB (that I know of either) that would allow for the handling of nested replicants easily.
Even though I haven't figured it out [yet] - I think there should be a way (using the same information) to create an algorithm that handles nested replicants correctly. However, instead of post-processing the list of all xpaths found, I think I would need to somehow do it at the point in time that I'm constructing each xpath - somehow creating not only distinct counters for each replicant (simple hash) but also being able to determine when to increment the base counter for the containing replicant and re-initialize the inner-counter for the nested replicant. At first blush, this doesn't sound too difficult either when you're only dealing with one level of nesting, but as soon as you start talking about multiple levels of nesting at different levels within the XML structure, it clearly is a more complex issue that requires a lot of careful thinking ... fortunately, for now, I'm spared the need to do so, but I may end up coming back to it - if not for my customer, for me.
Thanks - I'll let you (et al) know what happens with regard to the CGI encoding.

Adam Stoller

BINGO!
Adding the charset directive to the content-type header - and removing all the various encoding things I did before - appears to have fixed the issue.

print qq(content-type: text/html; charset=utf-8\n\n...);

No need for the Encode module, the HTML::Entities module, binmode, replacing the bullets, etc. I do explicitly ensure that the open of the source DCR is done with UTF-8 encoding, and now I make sure the output from the CGI is done in UTF-8 encoding - and that seems to take care of everything.

Thanks much - I was hoping it was something simpler than what I was doing, but had completely overlooked the CGI output stream (well, not totally, I tried futzing with binmode on STDOUT but either I did it wrong, or it had no effect).

ISCBorisB

BINGO!
Adding the charset directive to the content-type header - and removing all the various encoding things I did before - appears to have fixed the issue.
print qq(content-type: text/html; charset=utf-8\n\n...);
No need for the Encode module, the HTML::Entities module, binmode, replacing the bullets, etc. I do explicitly ensure that the open of the source DCR is done with UTF-8 encoding, and now I make sure the output from the CGI is done in UTF-8 encoding - and that seems to take care of everything.

Thanks much - I was hoping it was something simpler than what I was doing, but had completely overlooked the CGI output stream (well, not totally, I tried futzing with binmode on STDOUT but either I did it wrong, or it had no effect).

My pleasure