Submitted generated files can't be deleted/updated

I have a workflow originally developed on Solaris 8-TeamSite 5, tweaked for Solaris 9-TeamSite 6.5, that I'm now migrating to Linux-Teamsite 6.7.1, but having a problem that I can't figure out. The workflow is a generate - deploy to dev - approve - deploy to production - submit process. It works perfectly - once.

If I update the DCR, the workflow will not update the generated HTML. The workflow external task that does the deploys has a step that checks if the HTML file exists, and if it exists, issues an unlink just before issuing the iwgen. The unlink is failing for some reason.

When I try doing an rm on the file from the command line, I get a prompt that asks whether to "remove write-protected regular file". If I answer yes, the file deletes...so it doesn't appear to be a rights problem as well as much as a configuration setting...but I'm not sure what it is.

]$ ls -la test.shtml
-rw-rw-rw- 1 diwv999 iwglobal 1471 Jun 15 17:15 test.shtml
$ rm test.shtml
rm: remove write-protected regular file `test.shtml'? y
$ ls -la test.shtml
ls: test.shtml: No such file or directory

Once I delete the file manually, the workflow will work the next time, but the problem repeats once the workflow completes and the HTML is submitted.

Find more posts tagged with

Comments

nipper

A couple things, first do you have any aliases defined for rm ? Like rm -i ?

Also what are the permissions of the directory holding the generated file ? To delete a file you need write access to the directory not just the file

Finally, why remove the file first ? I always just overwrite.

HTH

me

mike_jaixen

Interesting... I noticed the files were not readable by the owner (which is our Interwoven account)

[diwv999@omhq10b1 WORKAREA]$ ls -la
total 2
dr-xr-xr-x 3 diwv999 nobody 512 Jun 13 09:34 .
drwxrwsr-x 5 diwv999 nobody 512 Jun 13 09:33 ..
d-wxrwxrwx 5 diwv999 iwglobal 512 Jun 15 16:31 main
[diwv999@omhq10b1 WORKAREA]$ chmod -R 775 main

However, that change didn't make a difference...

In fact, I noticed that when the file is submitted, the group gets changed from iwglobal and the tsgroup I've assigned to it over to group "dev":

[diwv999@omhq10b1 htdocs]$ ls -la
total 8
drwxrwxrwx 2 diwv999 iwglobal 512 Jun 18 11:14 .
drwxrwxrwx 5 diwv999 iwglobal 512 Jun 15 16:31 ..
-rw-rw-rw- 1 remp022 iwglobal 35318 Jun 7 14:16 index.shtml
-rwxrwxrwx 1 diwv999 iwglobal 53 Jun 15 16:30 interwoven.txt
-rw-rw-rw- 1 remp022 iwglobal 2952 Jun 7 14:16 re.shtml
-rwxrwxrwx 1 diwv999 iwglobal 1422 Jun 15 16:13 test2.shtml
-rw-rw-r-- 1 diwv999 dev 1471 Jun 18 11:14 test.shtml

nipper

Interesting... I noticed the files were not readable by the owner (which is our Interwoven account)

[diwv999@omhq10b1 WORKAREA]$ ls -la
total 2
dr-xr-xr-x 3 diwv999 nobody 512 Jun 13 09:34 .
drwxrwsr-x 5 diwv999 nobody 512 Jun 13 09:33 ..
d-wxrwxrwx 5 diwv999 iwglobal 512 Jun 15 16:31 main
[diwv999@omhq10b1 WORKAREA]$ chmod -R 775 main

However, that change didn't make a difference...

In fact, I noticed that when the file is submitted, the group gets changed from iwglobal and the tsgroup I've assigned to it over to group "dev":

[diwv999@omhq10b1 htdocs]$ ls -la
total 8
drwxrwxrwx 2 diwv999 iwglobal 512 Jun 18 11:14 .
drwxrwxrwx 5 diwv999 iwglobal 512 Jun 15 16:31 ..
-rw-rw-rw- 1 remp022 iwglobal 35318 Jun 7 14:16 index.shtml
-rwxrwxrwx 1 diwv999 iwglobal 53 Jun 15 16:30 interwoven.txt
-rw-rw-rw- 1 remp022 iwglobal 2952 Jun 7 14:16 re.shtml
-rwxrwxrwx 1 diwv999 iwglobal 1422 Jun 15 16:13 test2.shtml
-rw-rw-r-- 1 diwv999 dev 1471 Jun 18 11:14 test.shtml

Check your submit.cfg to see if that is changing the owner/permissions. YOu can use iwtestcfg

So recap, the change to the dir did not help ? Check the ownership of the files after they have been submitted.

Also, if you have a log file, dump all your env variables to it during the generate process.

Why are you deleting the HTML, will over-write not work ?

Andy

mike_jaixen

submit.cfg is changing the uid, but not the gid for this branch:

.* {
/$ {
uid=diwv999
perm=2775
}
.* {
uid=diwv999
perm=0664
}
}

Not sure why this was set up this way...this file hasn't been touched in 4 years. Same thing as to why we we delete the file instead of letting it overwrite; it was coded up this way before I took over tech support. In any event, the problem is the same. The iwgen step can't overwrite the file either.

I was hoping that the submit was grabbing the "dev" group from the branch (and I screwed it up initially), but performing an iwchgrp on the branch didn't make a difference.

nipper

2 things, run iwtestcfg on the file just to be certain that there is not anything else in submit.cfg that is changing the group.

ALso, I noticed you have the group stick bit set on the grandparent directory (drwxrwsr-x 5 diwv999 nobody 512 Jun 13 09:33 .. ). Any other dirs like that ? That will cause any new files/dirs in that directory to inherit the group from its parent.

Can you stop the WF between the generate and the submit ? It would be nice to see the exact state of ownership/permissions on the file after the script is run and before submit.cfg takes place.

mike_jaixen

Here's what iwtestcfg reports:
$ iwtestcfg /default/main/media_technology/WORKAREA/site_wide_assets/submit.cfg
Matched area pattern "^/default/main/media_technology/.*"
Matched file pattern ".*"
Actions to do are:
uid=diwv999
gid=diwvgrp000
perm=0664

The sticky bit is set on the WORKAREA directory...which I believe is created by TeamSite by iwmkbr.

Interesting...walking through this, submit seems to be a red herring. I noticed that the iwgen step is creating the test.shtml file with gid dev instead of the group for the rest of the files in the workarea. I added a usertask into the workflow and here's what I saw.

(Generate)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 remp022 dev 1471 Jun 18 14:59 test.shtml
(Userstep)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 diwv999 dev 1471 Jun 18 14:59 test.shtml

nipper

Here's what iwtestcfg reports:
$ iwtestcfg /default/main/media_technology/WORKAREA/site_wide_assets/submit.cfg
Matched area pattern "^/default/main/media_technology/.*"
Matched file pattern ".*"
Actions to do are:
uid=diwv999
gid=diwvgrp000
perm=0664

Can you run iwtestcfg on the specific file you are trying to generate, test.shtml ?

The sticky bit is set on the WORKAREA directory...which I believe is created by TeamSite by iwmkbr.

Maybe, I did not notice the prompt listed your path as WORKAREA.

Interesting...walking through this, submit seems to be a red herring. I noticed that the iwgen step is creating the test.shtml file with gid dev instead of the group for the rest of the files in the workarea. I added a usertask into the workflow and here's what I saw.

(Generate)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 remp022 dev 1471 Jun 18 14:59 test.shtml
(Userstep)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 diwv999 dev 1471 Jun 18 14:59 test.shtml

OK, so the external task is running as remp022/dev ? Can you confirm with the WFT ?

What permissions does remp022 have ? Correct ones to overwrite the file after submit ?

Are you logging in the external task ? Can you dump all the environment variables to your log ?

Adam Stoller

[...]
(Generate)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 remp022 dev 1471 Jun 18 14:59 test.shtml
(Userstep)
[diwv999@omhq10b1 htdocs]$ ls -la test.shtml
-rw-rw-r-- 1 diwv999 dev 1471 Jun 18 14:59 test.shtml

Try running:

) - I'm guessing that the gid associated with remp022 is probably "dev" - and thus that's why the groupid is being set to dev.

mike_jaixen

The gid (dev) is the gid associated with the user ID for both my personal id (remp022) and the Interwoven account (diwv999)

Problem is that this workflow appears to be dependent on the gid (in this case diwvgrp000) to be assigned to the files created by iwgen. diwvgrp000 is now set up as a tsgroup and is the group assigned to the workarea. dev is an OS group tied to my user ID and the system user ID.

Here's what iwtestcfg reports on the file (Sorry, I misunderstood what iwtestcfg did...)

$ iwtestcfg test.shtml
Matched area pattern ".*"
Matched file pattern ".*"
Actions to do are:
uid=diwv999
perm=0664

$

Here are my environment variables:
[11:28:29] 06/19/2007::Environment Variables:::IW_STORE::/iwcontent/iw-store
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

SH_CLIENT::::ffff:69.58.214.42 2647 22
[11:28:29] 06/19/2007::Environment Variables:::IWAUTH::52616e646f6d4956494ae1ab25269dcd3a5a450fc22ebac4bbedd6808bfb922320e810d7d9b29517cf06e2e46eef486d
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

SH_ASKPASS::/usr/libexec/openssh/gnome-ssh-askpass
[11:28:29] 06/19/2007::Environment Variables:: Smiley Surprised

RACLE_HOME::/software/oracle/cli-10.02.00.03
[11:28:29] 06/19/2007::Environment Variables:::LESSOPEN: Smiley Indifferent

/usr/bin/lesspipe.sh %s
[11:28:29] 06/19/2007::Environment Variables:::EXITVAL::0
[11:28:29] 06/19/2007::Environment Variables:: Smiley Tongue

WD::/software/interwoven/teamsite
[11:28:29] 06/19/2007::Environment Variables:::USER::remp022
[11:28:29] 06/19/2007::Environment Variables:: Smiley Tongue

OSIXLY_CORRECT::y
[11:28:29] 06/19/2007::Environment Variables:::G_BROKEN_FILENAMES::1
[11:28:29] 06/19/2007::Environment Variables:::LOGNAME::remp022
[11:28:29] 06/19/2007::Environment Variables:::RESULT::0
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

HLVL::5
[11:28:29] 06/19/2007::Environment Variables:::INPUTRC::/etc/inputrc
[11:28:29] 06/19/2007::Environment Variables:: Smiley Tongue

ATH::/usr/bin:/bin
[11:28:29] 06/19/2007::Environment Variables:::HISTSIZE::1000
[11:28:29] 06/19/2007::Environment Variables:::TERM::xterm
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

SH_TTY::/dev/pts/2
[11:28:29] 06/19/2007::Environment Variables:::HOME::/privdir/remp022
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

SH_CONNECTION::::ffff:69.58.214.42 2647 ::ffff:167.132.88.70 22
[11:28:29] 06/19/2007::Environment Variables:::TRACEFILE::/logs/interwoven/teamsite/iwtrace.log
[11:28:29] 06/19/2007::Environment Variables:::IW_RELBINDIR::/software/interwoven/teamsite/bin
[11:28:29] 06/19/2007::Environment Variables:::MAIL::/var/spool/mail/remp022
[11:28:29] 06/19/2007::Environment Variables:::HOSTNAME: Smiley Surprised

mhq10b1.uprr.com
[11:28:29] 06/19/2007::Environment Variables::lithium::/software/interwoven/teamsite/bin/iwutild
[11:28:29] 06/19/2007::Environment Variables:::LS_COLORS::no=00:fi=00:di=00;34:ln=00;36 Smiley Tongue

i=40;33:so=00;35:bd=40;33;01:cd=40;33;01 Smiley Surprised

r=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:
[11:28:29] 06/19/2007::Environment Variables:::IW_MOUNT_PRIVATE::/iwmnt
[11:28:29] 06/19/2007::Environment Variables:: Smiley Frustrated

****::/bin/bash
[11:28:29] 06/19/2007::Environment Variables:::KDEDIR::/usr
[11:28:29] 06/19/2007::Environment Variables:::IW_MOUNT::/iwmnt
[11:28:29] 06/19/2007::Environment Variables:::IW_HOME::/software/interwoven/teamsite

nipper

OK, got another Q, in the WFT, what user is the external task running as ? It looks like it should be your personal login (remp022).

mike_jaixen

The "generate & deploy" tasks are defined inside declare_task("Deploy_To_Dev",
name=>"Deploy_To_Dev",
description=>"Deploying to development. Please wait.",
owner=>"$iw_user",
type=>"externaltask",
command=>"$dev_cmd",
areavpath=>"$iw_workarea",
#areavpath=>"$iw_workarea");
timeout=>"+000060");

Owner is set to to the person who submitted the workflow from TeamSite.

nipper

OK, so that is why the group is set to dev.

you *could* try to change your primary group to another other than dev (hopefully one that has permissions to overwrite the file). I do not know if there is a way in Perl to do a newgroup command.

Lets take a step back, what group is required to overwrite the file you are talking about and are you a member ?

mike_jaixen

In this workarea, it's diwvgrp000. In other workareas, it may be any of the other groups in our system. I suppose I could just run iwchgrp after running iwgen and force the issue here...but I get the feeling that I've got something wrong with this old workflow. Perhaps it's an error that our original developer left when he developed this on Solaris. (Like manually deleting the file before running iwgen...)

As for group memberships:
$ iwgroup list-groups-of-user remp022
dev
diwvgrp000
iwEveryone
$ iwgroup list-groups-of-user diwv999
dev
diwvgrp000
iwEveryone
$

nipper

In this workarea, it's diwvgrp000. In other workareas, it may be any of the other groups in our system. I suppose I could just run iwchgrp after running iwgen and force the issue here.

I am not so certain that a iwchgrp will help. Can you add this group to submit.cfg ? That would be easier than iwchgrp.

mike_jaixen

I am not so certain that a iwchgrp will help. Can you add this group to submit.cfg ? That would be easier than iwchgrp.

Hmmm...every new branch would then require an update to submit.cfg to get the permissions set right. We probably set up a new branch once or twice a month (with over 100 branches already) which will make this rather ugly. :eek: :eek: :eek: :eek:

mike_jaixen

Going back and RTFM ( Smiley Very Happy

) on groups and access control, it looks like the model that was originally set up might have been overkill. It appears that in order to access any files, you first need access to the branch and WORKAREA... Access to branches and WORKAREAs can be managed by OS or TeamSite groups, but access to files apparantly cannot be managed by TS Groups.

So what I'm thinking is that I need to specify some sort of OS group as the primary group for all TeamSite users. With that in place, then any generated files will have a gid that can be accessed by any TeamSite user as long as their user ID appears in the group for that WORKAREA.

Am I getting warmer with this?

nipper

It sounds like you are close. does dev have access to the workarea ? What group does ?

mike_jaixen

Dev doesn't have access to the workarea...but the members of dev should also be members of the TS group that has access to the workarea.

nipper

What group owns the WA ? Just for fun change your primary group (in /etc/passwd) to that specific group and try the workflow.

mike_jaixen

The workarea is owned by a tsgroup, so even if I had access to /etc/passwd to change a group, I couldn't set the default group at the OS level to a tsgroup.

nipper

Can you change the owner of the WA to dev (just to test to see if we are on the correct track ) ?

mike_jaixen

Changed workarea to have gid dev with didn't have any impact.

Thinking back, it's interesting that the script runs the first time as user ID remp022/dev and runs again as remp022/dev. The submit filter changes the owner to diwv999, but leaves the gid alone. The file permissions are Group: ( dev ) Read and Write ...which doesn't make any sense. Either what Teamsite FS thinks group dev is different from what the workflow is running as, or something is royally mucked up.

nipper

Changed workarea to have gid dev with didn't have any impact.

Thinking back, it's interesting that the script runs the first time as user ID remp022/dev and runs again as remp022/dev. The submit filter changes the owner to diwv999, but leaves the gid alone. The file permissions are Group: ( dev ) Read and Write ...which doesn't make any sense. Either what Teamsite FS thinks group dev is different from what the workflow is running as, or something is royally mucked up.

Well, depends. Since you are doing the unlink in the perl, then the permissions of the parent directory are what is looked at, which is group iwglobal to do a remove.

I am at a lost to understand why the plain overwrite of the generated file did not work (with dev having write access to the workarea).

mike_jaixen

I spoke too soon...I resubmitted this workflow again a couple of hours later and now the file regenerates with the WORKAREA owned by dev (OS group). I'm betting that there might have been some cached information when I first tried it. The delete still fails (as you say, because the group "dev" doesn't have access to the files even though the user ID remp022 is a member of the Interwoven group), but it now overwrites the HTML file.