PERL Q.... (concurrency)

I have some triggers running when a file is copied to so some funky stuff.

Things are getting really funky, and not in a good way.

One time I noticed this not so good behavior (during my test) another user was also running the same script. So I am thinking that my perl is not capable of having serveral copies at once.

1st obvious place is the log file. I am sick of having wf_dd_12345.log so I have been using this:

open (LOG, ">$iwhome/tmp/Save_cp_deploy.log")

(with no error checking) So I can easily see that this puppy will croak.

So that is problem 1, are there things in perl that are global ? If I am passing info via ENV variables, that may be a problem.

thoughts, ?

Andy

Find more posts tagged with

Comments

Adam Stoller

I'm confused - what are you asking about here? temporary file naming conventions? file locking conventions? ways to pass information from one Perl program to another without using a temporary file?

Perhaps you can provide a bit more of a descriptive sceanario of a "clean run" and then add descriptions of where you see problems in concurrency and what you *want* to be able to do?

--fish
(Interwoven Senior Technical Consultant)

nipper

Sure

I was asking a more general question about Perl, on if there are any internal Perl issues with concurrency. The external ones (filename replaced with filename_$$) are easy,

So I have some scripts (both WF and formAPI initiated) that take a list of files, path, etc via ENV variable. These move specified files around and then kick off opendeploy or datadeploy. (roughly 2 minutes)

If 2 users save some info via template, & the scripts kick off, close to the same time. Are there a set of env variables for each child process or does the 1st get overwritten by the second ?

Obviously I need to run OD with the -inst flag.

Any clearer ?

Andy
Andy

Adam Stoller

If I understand your question correctly

there are two separate instances of the same script run
each instance reads from and potentially sets some environment variables
each instance kicks off another process
the other process will make use of environment variables that may have been set by the instance of the initial script which called it

Is that about right?

There should be no problem with cross-contamination of the environment variables. When your initial script starts out, it inherits environment information from its parent (e.g. the shell process) - if both instances of the script were run from the same shell, there is a very good chance they both started with exactly the same environment information. If they started from different shells, perhaps belonging to different users, then there is more potentential for them to have started with different environment information.

Once the script starts - that environment information is generally treated as "local" information. For example:

#!iwperl -w
my $log = "c:/temp/ugh";
if (!defined($ENV{I_RAN})){
    $ENV{I_RAN} = 0;
    unlink("$log");
}
my $user = $ENV{USERNAME};
$ENV{USERNAME} = "FishFace$ENV{I_RAN}";
if (open(OUT, ">>$log")){
    print OUT "($user)($ENV{USERNAME})\n";
    close(OUT);
}
$ENV{I_RAN}++;
`iwperl $0` if ($ENV{I_RAN} < 3);
if (open(OUT, ">>$log")){
    print OUT "($user)($ENV{USERNAME})\n";
    close(OUT);
}
if (open(IN, "<$log")){
    print while(<IN>);
    close(IN);
}
exit(0);C:\temp>echo %USERNAME%
ghoti
C:\temp>iwperl foo.ipl
(ghoti)(FishFace0)
(FishFace0)(FishFace1)
(FishFace1)(FishFace2)
(FishFace1)(FishFace2)
(FishFace0)(FishFace1)
(ghoti)(FishFace0)
C:\temp>echo %USERNAME%
ghoti

Yes, it's a simplistic example, but it shows that the environment information changed within the script has no bearing on the shell which started it, however it does have an effect on any other process started from within it.
In this way, your two distinct instances of running the same script, at the same time, could set/modify the same environment variable with different values and then call the same other script or program and the results are distinct.
The only place where the above example fails to work for that scenario, is the log file ($log) - but I can fix that with a few tweaks to the above code:

#!iwperl -w
my $DEBUG = $ENV{DEBUG_FOO} || 0;
if (!defined($ENV{I_RAN})){
    $ENV{I_RAN} = 0;
    $ENV{I_LOG} = "c:/temp/ugh$$";
    unlink("$ENV{I_LOG}");
}
my $log = $ENV{I_LOG};
my $user = $ENV{USERNAME};
$ENV{USERNAME} = "FishFace$ENV{I_RAN}";
if (open(OUT, ">>$log")){
    print OUT "($user)($ENV{USERNAME})\n";
    close(OUT);
}
$ENV{I_RAN}++;
`iwperl $0` if ($ENV{I_RAN} < 3);
$ENV{I_RAN}--;
if (open(OUT, ">>$log")){
    print OUT "($user)($ENV{USERNAME})\n";
    close(OUT);
}
if ($ENV{I_RAN} == 0){
    if (open(IN, "<$log")){
        print while(<IN>);
        close(IN);
    }
    unlink("$log") if (!$DEBUG);
}
exit(0);

Does that help answer your question? or did I just ramble on about this for no reason?

--fish
(Interwoven Senior Technical Consultant)

nipper

Fish

You definately were helpful. But I still have one lingering question. These scripts may take 2-3 minutes to complete, and we have 85 users rolling out at the end of the month, so it
is a real concern

I think I know the answer, but i woud like your opinion:

these scripts are kicked off via external task from a WF.
The WF passes infomation to the script via variables, just
wondering if there are possibilities of collisions there.

Andy

Adam Stoller

If each instance of the script is kicked off from a separate instance of a workflow - then there should be no colissions as you would have a distinct location (via the job / jobid) to store the information as variables within (if I'm understanding you correctly).

If you have multiple such scripts envoked from a single workflow - concurrently, and you are trying to use workflow variables to store the information from the various instances of the script - then yes, you will run into collision in the sense that one externaltask may set the job variable to one thing and another job variable would re-set it to something else.

This can be dealt with in a couple of ways:
1) create unique variable names for each instance of the script
2) [and I think you eluded to this - but I'm in a bit of a rush right now] - check to see if the variable exists before creating / setting it - if it does exist, retrieve the current value, append the new value and then set the variable again. This still has potential for conflict though - so you might need to do something fancy with a "lock" variable of some sort so that while one instance of the script is doing this, another instance of the script would be in a wait-loop... hmm - could be interesting, don't have time to think it through right now - sorry..

--fish
(Interwoven Senior Technical Consultant)