nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

TeamSite Slows down on workflow submit to Stage

Paul

We have an issue that has become quite frustrating We are using TeamSite 5.5 on an NT Server and whenever a workflow reaches the Deploy to Staging area within TeamSite, the server bogs down to a crawl.

If I, or any user, tries to edit a file, or view a file, the app just clocks away for minutes sometimes hours while that step in the workflow is being run.

This seems to be worse with different file types such as .doc, .xls, .PDF or even .txt files as opposed to .html or image files.

Has anyone experienced this and/or can anyone suggest a remedy?

Find more posts tagged with

Comments

Migrateduser

I haven't seen this in any version of TeamSite on any platform. Do you have any submit or other Interwoven triggers configured on the system? What does iwstat -c on the server show when the performance is bad? How many files are being submitted at once? How many jobs are active on the system?

Paul

Here's the output of iwstat -c (it's happening right now)

WDCINT51

C:\iw-home\bin>iwstat -c
Store Status
default Running
CIGNA_com Running
workflow Running

ID Thread User Duration Store Operation
0x6d5ae 0x6ab WDCINT51\ROOT 0.000 - GetServerStatus
0x6cdd4 0x42 WDCINT51\TSIMP_wdcint51 102.000 default CFILE_SETATTR_svc
(6a,41a7) 925c5/911ab
0x6cd74 0x6a1 SYSTEM 102.109 default SubmitFSE 0x000021
00000000000045e00

Cache Active Dirty To Purge Available
default 17589 404 0
CIGNA_com 148 0 0
workflow 2222 0 0
Total 19959 404 0 40041

There are 80 files in this job but I have seen it bog on 5 files. Currently there are 8 active jobs on the system

Migrateduser

So there are no submit triggers?

To tell you the truth this output doesn't help me much - I would probably file a case with support at this point. 102 seems like a large number for any operation - I wonder if there is some deadlock condition.

It looks like it's doing something with extended attributes (though it could be some other kind of attribute), but you said that the files were not templated - are you using MetaTagger (which might be doing something with the attributes) or using custom extended attributes somehow?

Have you checked the backing store for corruption lately?

13adam13

How often are you creating editions. I have found that each submit takes a little bit longer than the last, and after many submits, this actually becomes an exaggerated amount of time.

--Adam Wilson
Senior Programmer

Paul

I'm sorry John I was ini tyhe middle of answering when I got a phone call. We do have the step in the workflow that submits files to the staging area in TeamSite that is called by:

__INSERT__(&get_cmd("submit_all.ipl"));

submit_all.ipl looks like:

#########################################################################
#
# submit_all.ipl submits all files attached to the workflow and any DCRs
# related to those files to Staging.
#
#########################################################################
use TeamSite::WFtask;
use TeamSite::Config;
my $iwhome = TeamSite::Config::iwgethome();

my $task = new TeamSite::WFtask($ARGV[1]);
my $areavpath = $task->GetArea();

my @files;

if ($task->IsValid()) {
@files = $task->GetFiles();
foreach my $fileintask (@files) {

#########################################################################
#
# Find out if the file is a DCR or a generated file
#
#########################################################################
my $cmd = "$iwhome\\bin\\iwextattr -g TeamSite/Templating/PrimaryDCR $areavpath\\$fileintask";
my $out = `$cmd 2>&1`;

#########################################################################
#
# If not a DCR, then find out the corresponding doctype, dcrname
# and build up the relative path of the DCR to the area root and
# then attach it to the workflow task.
#
#########################################################################
if ( $out !~ /Attribute TeamSite\/Templating\/PrimaryDCR not found/ ) {
my $cmd2 = "$iwhome\\bin\\iwextattr -l $areavpath\\$fileintask";
my @eas = `$cmd2`;
my %attribs = ();

foreach (@eas) {
chomp ($_);
my ($name, $value) = split (/\=/, $_);
$attribs{$name} = $value;
}

my ($doctype, $dcrname);

foreach my $attrib_name (keys %attribs) {

if ( $attrib_name =~ /TeamSite\/Templating\/PrimaryDocumentType/ ) {
$doctype = $attribs{$attrib_name};
}

if ( $attrib_name =~ /TeamSite\/Templating\/PrimaryDCR/ ) {
$dcrname = $attribs{$attrib_name};
}

}

#########################################################################
#
# Piece together path to the DCR and submit it to Staging.
#
#########################################################################
my @lists = split(/\\/, $fileintask);
my $dcr = "\\templatedata" . "\\" . $doctype . "\\" . "data" . "\\" . $dcrname;

my $iwsubmit = "$iwhome\\bin\\iwsubmit -w -u $areavpath\\$dcr Submit";
my @results = `$iwsubmit 2>&1`;

}

#########################################################################
#
# Submit the actual file (generated or not) to Staging.
#
#########################################################################
my $iwsubmit = "$iwhome\\bin\\iwsubmit -w -u $areavpath\\$fileintask Submit";
my @results = `$iwsubmit 2>&1`;
}

$task->CallBack(0);

}

exit;

Paul

Adam we do a daily edition

Migrateduser

I don't see anything suspicious in this code. I imagine that one of the calls to iwextattr is pausing the script - would it be possible to step through the script with iwperl -d and see if there is one command that is taking a long time to run? or you might find that it's looping more than you would expect (I haven't looked at the code closely).

In general it's a good idea to wrap all CLT calls with a module that can be configured to log output, then you can enable/disable whether that output is logged (you might want to always log in dev but only when you are having a problem in prod).

Also you seem to be ignoring any error messages returned from the CLTs, not sure if that's what you want to do...

I still wonder if there's a submit trigger (this looks like a submit script). Can you run iwlsat and send the output?

13adam13

Looking at your iwstat -c output, looks like you have some pretty extensive workflows going on. I thought we had a lot, but are only using 247 bytes compared to your 2222.

Also, it looks like your total cachesize setting is pretty low. After doing a lot of research and tweeking, I found that our server operated best at 180000 cachesize, and you are running at about 20000.
I can see where having such a large active workflow cache along with such a small overall cache might be giving you performance problems. To change the cachesize, you would modify iw.cfg, and then you need to restart TeamSite(this is one of those few settings that a reset won't enable).

--Adam Wilson
Senior Programmer

Migrateduser

I don't know much about cache sizing but wouldn't that be a problem in general (not just at submit time)?

13adam13

I think that a submit is a pretty heafty operation for TS to do, and that is why you are seeing the problem when submitting. I know that when our cachesize was too small, and someone would submit, it would actually cause the TS server to quit responding and we would have to restart it.

This may be a problem always, but you just don't notice when a process that could take 5 seconds takes 8 or 9 instead.

Having said that, you may be right, and I am barking up the wrong tree.

--Adam Wilson
Senior Programmer