nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Workflow, index page, many DCRs, etc.

nipper

OK, so the good news is that if I am dealing with an issue like this then my implementation must be doing pretty well.

I have a page that reads several hundred DCRs and builds and index page. I only use 1 workarea and like to build this when a DCR is submitted. The workflow steps (that matter) are generateHTML, deploy to test, review, submit, production deploy.

The issue is when to build the index page and where to read from.

Currently I read the WA DCRs and build the index page in the generateHTML section of my workflow. The obvious issue is that unapproved changes may show up on the index.

If I read the DCRs from the Staging area, then the content being reviewed is not in the index and will not be until the next DCR is submitted.

So my thought is to read the DCRs from Staging unless the specific is attached to the job that the index page is. Since I am using iwpt_compile, that is not trivial because the TPL knows nothing about the job.

Other options are to do a generateHTML after submit (for that index page) but then the reviews happen with the updated index. Not stellar.

I could just process the index page every night, hour, etc, so that is is always out of date a bit.

Opinions ? Thoughts ?

Find more posts tagged with

Comments

moonkhanna

If you currently are not using a PT associated to the Data-type, You will need to write a PT for the solution.

Build your index page using this new PT and not with some custom scripting. In your PT, you can refer to this new DCR which is being read separately from the DCRs in Staging.

You know the rest.

If in case, I have not been understand your issue well, please elaborate

moonkhanna

If you currently are not using a PT associated to the Data-type, You will need to write a PT for the solution.

Build your index page using this new PT and not with some custom scripting. In your PT, you can refer to this new DCR which is being read separately from the DCRs in Staging.

You know the rest.

If in case, I have not been understand your issue well, please elaborate

I implemented the same solution with the above however you can also use custom scripting to accomplish the above. When you dirwalk through your DCRs in staging, also filter the currently editted DCR out.

ISCBorisB

If you currently are not using a PT associated to the Data-type, You will need to write a PT for the solution.

You either did not read Andy's post at all or misunderstood it completely

moonkhanna

You either did not read Andy's post at all or misunderstood it completely

Why do you think so? Did the solution I provided does not work in the scenario. Please elaborate?

nipper

I implemented the same solution with the above however you can also use custom scripting to accomplish the above. When you dirwalk through your DCRs in staging, also filter the currently editted DCR out.

Bo is right, you did not understand my question

I have 1 index DCR (choose the picture for the index page etc) that reads several hundred other DCRs for more content. The question how to mix using Staging DCRs for more of the data but workarea DCRs for any data currently in the approval process.

ISCBorisB

Bo is right, you did not understand my question

I have 1 index DCR (choose the picture for the index page etc) that reads several hundred other DCRs for more content. The question how to mix using Staging DCRs for more of the data but workarea DCRs for any data currently in the approval process.

I think it's a chicken and egg problem, there is no perfect solution.

I've implemented auto index-generation in several projects, for what it's worth here is the gist
of the (evolved) approach. Sure, it's a compromise. I use PR below as a Type metaphor, could be anything.

1. PR has two presentations, let's say pr.pt and pr_index.pt. Only pr.pt is registered in templating.
pr_index does not have a dedicated DCR ( Or rather, all of them are )

2. Submission Event is intercepted. Event Handler checks Files and if PR *Pages* are in the list starts PR Index generation
in asynch mode.

3. Index Generation script uses iwpt_compile with DCRs from STAGING and pr_index.pt. Output Files are generated into
dedicated zz_<something> WA created exclusively for various auto-generations. Upon successfully generation assets are
submitted (There is a caveat here! Submission gets intercepted. So, if my "zz_..." Workarea is the event source, Handler
should ignore it). Upon submission, iwupdate is used to get new assets to *regular* workareas.

My compromise here is that non-approved PRs are not indexed.

moonkhanna

Bo is right, you did not understand my question

I have 1 index DCR (choose the picture for the index page etc) that reads several hundred other DCRs for more content. The question how to mix using Staging DCRs for more of the data but workarea DCRs for any data currently in the approval process.

Andy, your initial question stated "I have a page that reads several hundred DCRs and builds and index page. I only use 1 workarea and like to build this when a DCR is submitted. The workflow steps (that matter) are generateHTML, deploy to test, review, submit, production deploy."

And that is why I went that direction. Anyways, in the above scenario, what I would have done is to categorise my data as a first step. So how many types of data-types you have? i.e. Is this a single data-type for which you are building an Index page for?

How many simulateous updations of the DCRs are there?

Depending on the complexity, a mix of locking mechanisms, User training and technical design would give you your solution..

moonkhanna

I think it's a chicken and egg problem, there is no perfect solution.

I've implemented auto index-generation in several projects, for what it's worth here is the gist
of the (evolved) approach. Sure, it's a compromise. I use PR below as a Type metaphor, could be anything.

1. PR has two presentations, let's say pr.pt and pr_index.pt. Only pr.pt is registered in templating.
pr_index does not have a dedicated DCR.

2. Submission Event is intercepted. Event Handler checks Files and if PR is in the list starts PR Index generation
in asynch mode.

3. Index Generation script uses iwpt_compile with DCRs from STAGING and pr_index.pt. Output Files are generated into
dedicated zz_ WA created exclusively for various auto-generations. Upon successfully generation assets are
submitted (There is a caveat here! Submission gets intercepted. So, if my "zz_..." Workarea is the event source, Handler
should ignore it). Upon submission, iwupdate is used to get new assets to *regular* workareas.

My compromise here is that non-approved PRs are not indexed.

ISCBorisB - this is the same thing, I proposed initially. Were you poitning to something else? As far as approvals are concerned, this will be a mix of User Training and Technical Design as I proposed in my previous post

moonkhanna

Here is second approach - get the stuff approved with an index page which is built from all the DCRs in Staging + the DCR in the current workflow.

Do a second generate of the index page, after the approval and Submission to IWOV Stage. This time with only the DCRs from Staging.

nipper

I think it's a chicken and egg problem, there is no perfect solution.

Yea that was my thought too, even if this was completely dynamic (not flat HTML) the fundamental issue still is there. Personal workareas and DAS would make it easier, but that is not an option.

I've implemented auto index-generation in several projects, for what it's worth here is the gist
of the (evolved) approach. Sure, it's a compromise. I use PR below as a Type metaphor, could be anything.

1. PR has two presentations, let's say pr.pt and pr_index.pt. Only pr.pt is registered in templating.
pr_index does not have a dedicated DCR ( Or rather, all of them are )

2. Submission Event is intercepted. Event Handler checks Files and if PR *Pages* are in the list starts PR Index generation
in asynch mode.

3. Index Generation script uses iwpt_compile with DCRs from STAGING and pr_index.pt. Output Files are generated into
dedicated zz_<something> WA created exclusively for various auto-generations. Upon successfully generation assets are
submitted (There is a caveat here! Submission gets intercepted. So, if my "zz_..." Workarea is the event source, Handler
should ignore it). Upon submission, iwupdate is used to get new assets to *regular* workareas.

My compromise here is that non-approved PRs are not indexed.

Hmm, sounds like a cleaner version of my idea to move the generate after the submit process. The downside is that the reviewer does not see the changes in theindex page during the review process.

ISCBorisB

ISCBorisB - this is the same thing, I proposed initially. Were you poitning to something else? As far as approvals are concerned, this will be a mix of User Training and Technical Design as I proposed in my previous post

Ah... Now I understand...

>>Build your index page using this new PT and not with some custom scripting.
>>In your PT, you can refer to this new DCR which is being read separately from the DCRs in Staging

It's absolutely same solution, sure...

Please note though that my solution is ALL custom scripting (Event Handler + IndexGenerator + Multi-DCR PT).
Btw, PT there is actually superficial and kept only for consistency and support. I might just as well have parsed DCRs
in "index genreration" Script and output the result. There is no "index DCR" whatever, new or otherwise etc...

But, sure, it's absolutely same solution

moonkhanna

Ah... Now I understand...

>>Build your index page using this new PT and not with some custom scripting.
>>In your PT, you can refer to this new DCR which is being read separately from the DCRs in Staging

It's absolutely same solution, sure...

Please note though that my solution is ALL custom scripting. PT there is superficial and kept only for consistency and support.
I might just as well have parsed DCRs in "index genreration" Script and output the result. There is no "index DCR" whatever, new
or otherwise etc...

But, sure, it's absolutely same solution

Yes Index DCR is missing. Thanks for pointing that out :-)

ISCBorisB

The downside is that the reviewer does not see the changes in theindex page during the review process.

Yes, absolutely. I've tried to err on the side of caution

nipper

Here is second approach - get the stuff approved with an index page which is built from all the DCRs in Staging + the DCR in the current workflow.

Do a second generate of the index page, after the approval and Submission to IWOV Stage. This time with only the DCRs from Staging.

Yea, that is an option. Along these lines I was thinking about: generate index, after the submit, a regenerate of the index (from staging only) submit again and deploy. The issue with that is then the index potentially goes out different than what was approved.

The idea of, when I run iwpt_compile, it looks for the job ID that the index DCR is attached to, and uses workarea DCRs for those and staging for the rest, is where I am leaning.

Is there any way to pass a variable through iwpt_compile ? I see the tag specific args, can I use those to pass something i may need ?

nipper

The idea of, when I run iwpt_compile, it looks for the job ID that the index DCR is attached to, and uses workarea DCRs for those and staging for the rest, is where I am leaning.

Is there any way to pass a variable through iwpt_compile ? I see the tag specific args, can I use those to pass something i may need ?

So reading perldoc teamsite: Smiley Tongue

t::iw_pt I can pass user defined variables to the PT.

So my plan is, in the generateHTML step of my WF, run iwpt_compile of my index page with something like this

iwpt_compile -pt pt -iw_pt-arg arg1="DCR1" arg2="DCR2" and collect that information with {iw_value name="$iw_value{arg1}"/}

That will be my list of files to use the WA version, the others will be Staging.

moonkhanna

So reading perldoc teamsite:t::iw_pt I can pass user defined variables to the PT.

So my plan is, in the generateHTML step of my WF, run iwpt_compile of my index page with something like this

iwpt_compile -pt pt -iw_pt-arg arg1="DCR1" arg2="DCR2" and collect that information with {iw_value name="$iw_value{arg1}"/}

That will be my list of files to use the WA version, the others will be Staging.

Great Finding Andy!
The thread seems to be good information for consultants intending to build index pages...I am rating the thread with 5 stars :-)

ISCBorisB

So reading perldoc teamsite:t::iw_pt I can pass user defined variables to the PT.

So my plan is, in the generateHTML step of my WF, run iwpt_compile of my index page with something like this

iwpt_compile -pt pt -iw_pt-arg arg1="DCR1" arg2="DCR2" and collect that information with {iw_value name="$iw_value{arg1}"/}

That will be my list of files to use the WA version, the others will be Staging.

Unless I've missed something here...

nipper

So I have a separate index DCR. It also reads a whole lot more detail DCRs. The index DCR defines keywords, images, intro etc on the index page. So I run iwpt_compile on the index which reads 1000 other DCRs. Net result is that the index PT does not know which of the 1000 other detail DCRs were attached to the same job as the index (and thus should use the WA not Stage version) I plan to pass the list of details DCRs with the arg string.

I hope that is more clear, though, after reading it, I can say it is probably not. What can I say, there is a reason I am a geek.

ISCBorisB

So I have a separate index DCR. It also reads a whole lot more detail DCRs. The index DCR defines keywords, images, intro etc on the index page. So I run iwpt_compile on the index which reads 1000 other DCRs. Net result is that the index PT does not know which of the 1000 other detail DCRs were attached to the same job as the index (and thus should use the WA not Stage version) I plan to pass the list of details DCRs with the arg string.

I hope that is more clear, though, after reading it, I can say it is probably not. What can I say, there is a reason I am a geek.

Perhaps, but why arg? Why not standard iw_pt-dcr? ( Other than geek-ness of course Smiley Happy

nipper

Perhaps, but why arg? Why not standard iw_pt-dcr? ( Other than geek-ness of course

I see what you mean, so I could pass the index and the other DCRs at the same time in iw_pt-dcr.

I think that might be a bit more confusing, not that any of this is truly clean. The other advantage is this gets added at the end, since my generateHTML script uses 1 iwpt_compile for 30 different template type, I can just set another variable and leave it blank at the end.

Either would probably work.

Andy

ISCBorisB

I see what you mean, so I could pass the index and the other DCRs at the same time in iw_pt-dcr.

I think that might be a bit more confusing, not that any of this is truly clean. The other advantage is this gets added at the end, since my generateHTML script uses 1 iwpt_compile for 30 different template type, I can just set another variable and leave it blank at the end.

Either would probably work.

Andy

Ahh, sure... If that's a changable part of 30+ calls it makes sense to keep it segregated.

Btw, I try not to pass too many DCRs but rather use 'iwpt_load_dcr' within multi-DCR PT. The reason is that the maximum
command length is not that great (8100 something on Windows)

nipper

Ahh, sure... If that's a changable part of 30+ calls it makes sense to keep it segregated.

Btw, I try not to pass too many DCRs but rather use 'iwpt_load_dcr' within multi-DCR PT. The reason is that the maximum
command length is not that great (8100 something on Windows)

Good point, I will have to test it with a bunch. I usually only submit max of 100 DCRs at a time and since I am only passing the DCR name (they are unique) I should be fine.

I certainly appreciate the help.

Andy

Adam Stoller

If you have a DCR that contains specific information related to the index page you are generating, load just that DCR from the command line, otherwise don't load any from the command line -- read them in as needed within the PT/Script code when you recurse through the directory. Why? Because if you try to put them all on the command line you can run into system errors resulting from either too long a command line or too many arguments on the command line, whereas if you process the directory, and task information [below] internal to the PT/Script process - you don't have that problem.

If you're going to pass a parameter to iwpt_compile.ipl - I suggest it be the taskid that kicked off this process - from which you can get the list of DCRs that need to be included from the workarea as opposed to those the rest which will be pulled in from the staging area - and if the parameter isn't present - just get everything from the staging area. (You indicated this earlier in the thread and then seemed to have either forgotten or abandoned this idea with no indication of why)

So - first time through - you have modified content being sent for review - you generate the index using the files in staging except those that are attached to the job, which you get from the workarea. The second time through, assuming everything has been approved and thus submitted, you re-generate the index based only on files in the staging area, and re-submit the generated index page.

It wouldn't surprise me if there's a bit more detail to be worked out here, but the above seems to be what you were dancing around throughout most of this thread and it seems reasonable ...

nipper

To quote Adam, *sigh*

This works now with the list of DCRs attached to the job (just filenames which are unique over the dirs). So unless I submit several hundred I am OK. Yea passing the task would be better and I may change to that. I say *may* for 2 reasons. 1 I am lazy and 2 if it works, why should I fix it ? That being said, I do get to work on Christmas eve, so maybe that is a time to do it.

Andy