iwgen command hangs while generating html pages

Hi ALL,

I am running IWOV version 6.1on Windows Platform. I am generating DCRs and final HTML files using scripts that are executed from the CGI. To generate the final HTML file, I am using iwgen command.

I am generating a DCR and a HTML file one for each month for a given year. So, when I try to generate all 12 html files in a loop, CGI script is hanging forever. It does create couple of html files and but hangs after a random of iterations.

I have opened a support case with IWOV. They were able to replicate the problem on their systems. However, till now not able to resolve the issue (Over 2 weeks due). The only suggestion they gave me was to use System($iwgencmd) in place of `$iwggencmd`. For some reason they say the back ticks takes up lots of system resources.

This seems so basic and yet not able to get it working. I could really use some hints, pointers in getting this issue resolved. Any ideas ? please post

Thanks,
Murthy

Find more posts tagged with

Comments

Migrateduser

Do you think this is specific to 6.x or can you recreate this in 5.x?

I think backticks taking more resources is total BS. It is the forking of the process that consumes resources. Plus you really should be capturing STDERR and STDOUT from these commands as error conditions, so whoever is spouting this rubbish is not advocating best practice (since I don't think you can capture STDERR using system()). Which means you should be calling iwpt_compile.ipl instead of iwgen/iwregen as they just hide the bugs and errors.

I think I remember seeing something about this related to Apache buffering, only on Windows. Are you on Windows? I think if you check the Apache error logs right after you get the problem you may see some helpful info. When you call the command line (wouldn't it be nice if it was an API), make sure you trap stderr with 2>&1, so it doesn't go to the Apache logs.

PankajK

Other than capturing STDOUT and STDERR, as suggested by john, you can try using iwpt_compile.ipl in place of iwgen inside the backticks to generate output pages.

peri

Thanks for your suggestion. I could not test it on 5.x environment ( I dont have an environment to test). This is certainly happening on 6.1.

However, I found strange behaviour. Per your suggestion, I made these changes and it started working. Dont see what the difference is though..

Initial Code: Does not work
$iwgencmd = "$iwhome/bin/iwgen -t \"$lsdPtDir/lsdarchive.tpl\" -r $dfile $param ";
`$iwgencmd`;

Changed Code: Works
$iwgencmd = "$iwhome/bin/iwgen -t \"$lsdPtDir/lsdarchive.tpl\" -r $dfile $param > c:\\temp\\out 2>err";
`$iwgencmd`;

the out file in the c:\temp folder is empty in the second case. I just dont understand as how redirecting the output to a file resolves the issue.

Any thoughts ?

Murthy

Dwayne

Just a hunch, but my guess is that it isn't the redirect of STDOUT that's fixing things, but the redirect of STDERR. By using backticks (`) you're already, in effect, redirecting STDOUT and then throwing it away. But the second example also redirects STDERR to a file called err. My guess is that this file is not empty. (You might want to put a full path to that file, btw).

--

Current project: TS 5.5.2/6.1 W2K

Migrateduser

I have seen a problem (but only from the GUI) where a presentation template that threw warnings during the iwpt_compile.ipl would cause the iwpt_compile.ipl process to hang and never complete. Possibly a similar thing is happening here with stdout (of course, then you would expect the file to have something in it)?

peri

The redirect file c:\\temp\\out.log is empty. But the pages are being generated with out any errors. I am not sure whether I should drop this thing or continue to follow-up with IWOV support.

Thanks for all your help
Murthy

Migrateduser

I think this is actually a bug in Apache on Windows - if your process writes too much data to STDERR, it seems to fill some sort of Apache buffer. I don't know exactly what happens from that point, but basically the process doen't work right once that buffer is full. You might check apache.org or other support for Apache; there may be a patch or workaround. Redirecting STDERR is probably good enough in your specific case, but lots of people have this problem and it would be nice to see a better solution. Checking the apache logs when any CGI process burps is a good place to start.

You might want to check your system for some hung processes - I am not sure if Apache just blocks procs once that buffer or whatever is full.

Extremely interesting that Interwoven support would not know about this issue which affects dozens of customers...

Edited by john on 10/20/04 12:20 PM (server time).

Migrateduser

Actually, that may not be exactly what you want to do. I think I ususally do something like:

my ${result} = `$iwhome/bin/iwgen -t "$lsdPtDir/lsdarchive.tpl" -r $dfile $param 2>&1";

if ( ${result} =~ m#..# )
{
# ${result} contains a generation error message, handle it.
}

Also I highly recommend iwpt_compile.ipl over iwgen/iwregen - in fact I wish they wouldn't distribute those binaries as they don't trap various bugs and errors. And if you're generating from a workflow extrernaltask under Windows, Interwoven has really messed up redirection in that environment (basically I think you have to use a temp file instead of memory to trap errors).

Fish?

Adam Stoller

I'm not sure what you want me to contribute here - iwgen and iwregen serve a purpose in that they take care of setting the TST EAs on the output file specified in the command line - they also, however, check against the templating.cfg file and won't let you use a DCT type or PT that isn't specified in the templating.cfg nor allow you to generate a file with a different extension that was specified in templating.cfg

As for the redirection stuff - yes - if you're using back-ticks, (or qx()) - you want to capture the results in a variable. If you are using system() or exec() you should probably direct the output to a file (and yes, use a full path for the file specification).

Another thing you can do in your CGI is:
use CGI::Carp qw(fatalsToBrowser);
which helps to throw error messages into the CGI window so you don't have to hunt around for the logfiles that those error messages might have been sent to when something goes wrong with the CGI script.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Migrateduser

> I'm not sure what you want me to contribute here

I think what you just wrote added value to this thread.

Cheers,

-John