nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Performance of Regenerate All By PT

System

This implementation has a custom menu item that regenerates all pages generated with a given PT. There is a performance problem with this CMI that appears to be related to scanning the Interwoven filesystem and executing iwextattr for each html file, iwregen for each match. I am not sure but it seems like performance would be better (one less fork) if the script called iwpt_compile.ipl directly (since I imagine iwregen is just calling iwpt_compile.ipl without adding much value, probably after checking extended attributes again and maybe even parsing templating.cfg). Best would be if the functionality of iwpt_compile.ipl was in a module so I could call the function instead of forking iwpt_compile.ipl, but really I think the problem is with the scan, not the generate (since there are not too many pages generated from each PT).

Is there any other way to improve the performance? I was thinking of storing a list of the files generated from each PT somewhere (in a file, as extended attributes on the PT, in a database, etc.) so it wouldn't have to run iwextattr for each file, but that would be customization I would prefer to avoid if possible.

The script is pretty basic but I don't think I can post it. Just looking for general performance suggestions specific to the problem of regenerate.

Find more posts tagged with

Comments

Johnny

I can tell you of a custom hack I did once that greatly improved things.

If you look at iwpt_compile.ipl it merely checks all params and calls a function in TeamSite: Smiley Tongue

Tparser::compile_page.

If you are comfortable with your knowledge and are happy to deal with any changes that might come in iwpt_compile (I havent had an indepth look at TST in v6.0 but I hear not much has changed), you could put this script into your module that you would call directly. Your module could even handle other actions such as setting the EA's.

I found a huge performance gain when generating with a large iteration. The support for it will be in your hands, but if you do it well, your call to the compile module wouldnt look that different to the system call. To be extra safe, if something ever breaks and you go this way, you could always reimplement the system call inside the module so that you wouldnt need to change any other code. I have never had any problems though.

I can send you the module if you like, take a look at iwpt_compile, you will see there isnt much to it, it wouldnt take you very long to fashion something to your needs.

I also find TeamSite: Smiley Very Happy

irwalk to be a little faster than the File::Find for searching through files.

John Cuiuli

Migrateduser

That would be great if you could send some sample code doing this with dirwalk and a module call. mailto:interwoven@jpw3.com. Also since it's a CMI (and I don't want the browser to timeout) I would want to regenerate on each hit or otherwise send something to keep the HTTP connection alive, not wait until all matches are found or otherwise let the browser timeout.

What about error checking? I usually do this by checking for any output from iwpt_compile or any non-zero return code. Can I do something similar with the module?

Thanks,

-John

Edited by john on 11/17/03 03:13 PM (server time).

Johnny

I dont have the code here now, I can send it tomorrow.

You could keep flushing status output as you go if you need to keep the connection alive.

When PERL 5.8 comes out we can finally use threads for stuff like that.

Error checking is the same. Any templating errors are sent from the PTparser modules.
Any param checking from iwpt_compile should be moved into your module.

The aim is to mimic it exactly, so the only difference is that it is a module, saving the need for a system call.

It would be nice if they were already available by IWOV, but I guess the CS API is there for all that now. We just have alot of code to port to get the benefits.

John Cuiuli

Johnny

If the response time is going to be large, you could simply just return with a message saying the job is kicked off.

The script can continue to run and the results could be emailed to the user once completed.

Just an idea.

John Cuiuli

JonathonG

In the short-term, just to improve performance, you could eliminate the calls to iwextattr and just call iwregen on every file. If the file isn't templating generated, it will error out, but that won't stop the process. Thus, instead of 1 call for every file (to get EAs) plus another call for each templating-generated file (regen, iwpt_compile, etc.), you would have 1 call for every file (iwregen).

I did this somewhere, and, by my recollection, it made a significant performance improvement.

Jonathon
Independent Interwoven Contractor

Johnny

I think the requirement is to generate a certain PT/template type.

Which means you'll have to check the file's template type first

John Cuiuli

Migrateduser

I also have to pass some custom parameters in @ARGV (which I think amy not be supported by iwregen?). But it still could be a good idea for systems without custom iwpt_compile.ipl @ARGV if most files matching a given regex are templated.

Migrateduser

I don't know if it's true but I remember hearing in college that two of the most expensive functions on a computer are opening a database connection and forking a process. Lately I think these have both been exceeded extremely by "new browser window" for which I think Interwoven must get some kind of kickback from Microsoft as it happens several hundred times any day I use TeamSite Templating. I wish someone at Interwoven engineering understood how important these performance issues are in the field.

It seems presentation templates are probably always going to be written in Perl (and anyway all of the other options such as cscript and xsl seem to require additional temp files and forks from PTs), and although it may be wrapped with command line tools, OpenAPI, ContentServices, or some other paradigm, the method for invoking them will probably always be iwpt_compile.ipl. While I think it's important that Interwoven distribute a version of Perl (and DBI/DBD) with a copyright/build date at least in this mil·len·ni·um, I would actually rather that the Interwoven C API be wrapped with Perl without the need for forking.

Johnny

So long as PERL is still around... (cant see it changing for TST/WF for a while), then I agree that a more complete/efficient API should be available.
It would be of great benefit.

It seems that there is always some sort of involved customisation done and iterative calls to commands like iwextattr and iwpt_compile can leave users waiting around.

I understand that alot of the development CLT are CLT and wrapped in PERL to make them more portable, but seeing these CLTs have to be compiled anyway, PERL/XS equivalents would be such a great help to our developments.

I am a little confused with the CS API as I believe it is quite complete. But I dont see how I can tie it into templating and workflow.

John Cuiuli