nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

IW functions in iwperl ?

nipper

So I am doing some massive EA setting and checking. Reading fish's perl tips, I would be better off if there was a perl module that I could call for iwextattr rather than back-ticking it.

Also, fish, if you are on-line, did you ever do any benchmark testing of find vs perl ?

my choice is:

find /iwmnt/branch/workarea -type f -exec check_ea.ipl {} \;

vs. putting the logic into check_ea.ipl for a -R option

If I benchmark this, I will post the results.

Andy

Find more posts tagged with

Comments

Adam Stoller

Sorry - I was out of touch for about a week, then unable to catch up with this forum when I was out of touch for another week - I'm finally starting to catch up (this is the last of the forums I had to catch up with - whew;-)

No - I haven't benchmarked find vs. using recursion inside the script - you could also try comparing combinations - like where you run find on directories, and then have the perl script process only files within that directory.

I'm not sure if it's in TeamSite 6 or coming in a later release - but I believe there are plans to provide a better mechanism for interacting with EAs such that you won't have the process overhead of invoking iwextattr for every attribute you want to set on every file... I can't provide details (mostly because I don't really know them - but also because it may not be committed to a release and/or the details may not be finalized) - but have faith, improvements are coming ....

--fish
(Interwoven Senior Technical Consultant)

laj1

my choice is:

find /iwmnt/branch/workarea -type f -exec check_ea.ipl {} \;

vs. putting the logic into check_ea.ipl for a -R option

I think that doing it inside perl will be faster.

You described the EA work you're doing as massive, so I'm going to predict that the file system traversal is going to amount to less runtime than the ea setting. IWOV has provided a nice little file system traversal module, which simplifies your life quite a bit. Plus, if you user the find(1) command, you have to fork and exec perl for every file you run check_ea.ipl on. If you do it in perl, you only fork and exec the perl executable one time.

If you think that your EA task is going to take a long time to run, then you might be interested in doing this rough bit of benchmarking.
If you expect a short runtime, then I'd say it isn't worth the effort to benchmark, although it would be interesting.

A reasonable back-of-the-envelope calculation would be to run the find command, with maybe the -print option to get a baseline time for directory traversal. Multiply that by the amount of time it takes (on average) to run check_ea.ipl on one file.

Now take check_ea.ipl, and add one line of code so that the first thing it does is exit. Time this. Now you have a rough idea of how long it takes to fork, exec and parse your program. Multiply that by the number of files you need to process, and you get the overhead of using find -exec.

Create perl script with the DirWalk module, that mimic the find -print from before. Run it and time it.

Now you have rough ideas of the time each component of each method will take. A few multiplications, and you have rough numbers to compare.

The question is really are we talking about saving minutes, hours or days, how often the task is repeated, and how big a percentage savings that is.

Who will care if you run for 25 minutes instead of 18 minutes? But running for 45 minutes instead of 4 hours is a very nice win.

Len.

Len Jaffe
My Heart Is A Flower
Update your DevNet profile - let us know who you are!

nipper

> Create perl script with the DirWalk module, that mimic the find -print from before. Run it and time it.

What is the DirWalk module ? I have not seen it.

I did some benchmarking, with 3 ways of doing it. I have about 13K templates. I did an open to see if it is valid XML (it was created programattically).

To parse the structure with find took about 28 minutes. I downloaded the recurse module from cpan.org, was much faster, but core dumped on large directories. THen I tried the
ls -R (done in IW code) dropped to about 7 minutes.

Can I download DirWalk from cpan ?

Andy

Migrateduser

It comes with TeamSite

$iwhome/iw-perl/lib/perl5/site_perl/5.005/TeamSite/Dirwalk.pm

=head1 NAME

TeamSite: Smiley Very Happy

irwalk - module for walking a directory structure

=head1 DESCRIPTION

This module is similar in spirit to File::Find, but it's more
flexible, faster, and object-oriented.

--
Jed Michnowicz
Interwoven Technical Consultant