nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Best policy for obsolete users owning files in TS workareas

HurricanesFan

Hello,

wanted some opinions about this and about people's preferences for file/dir ownership policies in TeamSite workareas - I expect there will be differing ones so it could be interesting.

In our TeamSite workareas we have many files/dirs owned by obsolete users eg:

-rw-rw-r-- 1 3013 nlmweb 1242 Jun 13 2012 robots.txt

drwxrwxr-x 2 3013 nlmweb 512 Nov 16 2013 sitemessages/

lrwxrwxrwx 1 3013 nlmweb 14 Feb 10 2005 symposium -> hmd/conference/

...and I am convinced this slows TeamSite in Linux due to issues with Linux user lookups of unknown users (anyone else think this?).

I have decided to change us from many individually-owned workareas to one group-owned workarea, so I can use submit.cfg to gradually change ownerships to generic tsmaster:nlmweb, but I note that although that works well for files it doesn't necessarily change directory ownerships. Note that we use straight Unix groups not "TeamSite groups".

Do people agree that is the best policy?

I was also looking at running iwidmap, but am edgy at running mass-changes on the content store unless there is a desperate need to do so. Besides my test attempt to run for just 1 user failed with:

./iwidmap -u -c 3013 tsmaster /usr/nlm/iwdata/iw-store/default

Unable to open TeamSite group file /usr/nlm/iw/iw-home/TeamSite/conf/tsgroups.xml

...assume tsgroups.xml is for "TeamSite Groups", which we don't use so why is it needed for this?

(BTW it was Support who suggested iwidmap as they thought unknown users might be the cause of an issue we have removing roles for certain users in certain branches:

iwaccess remove-permission-entry /iwmnt/default/main/nihseniorhealth -user nlmoccstest1 -role admin

Invalid or no permission entry

...although it shows this user is an admin in that branch. Anyone seen this before? Don't think iwidmap will solve this issue anyhow).

Cheers

Find more posts tagged with

Comments

nipper

Definately, you are on the right track.

Regardless of individual or shared workareas, files shoudl never be owned by Andy, they should be owned my your TS master account (tsadmin, iwmaster, whatever you happen to call it). You will need to be precise with grous and permissions to make certain everyone has the access they need.

Andy

HurricanesFan

nipper wrote:
Definately, you are on the right track.

Regardless of individual or shared workareas, files shoudl never be owned by Andy, they should be owned my your TS master account (tsadmin, iwmaster, whatever you happen to call it). You will need to be precise with grous and permissions to make certain everyone has the access they need.

Andy

Hmmm. In our case 'tsmaster' is a bit of a misnomer because it isn't actually a master account, and doesn't have an actual login (or has been lost in time; it was set up to own workflow Jobs).

Would it be best to set up a brand new generic login account as Master and use that? (and have each outgoing master whisper the password into the ear of the incoming master as they leave)

Would you recommend running iwidmap or let the system heal over time using the settings applied in submit.cfg (with maybe a few mass submits to help it along)?

Cheers

nipper

Doesn't have to be tsmaster, could be projectmaster, if you run mulitple projects on the same server.

You certianly can use the existing one, no one needs to log into it, just need a placeholder that will stick around. Doesn't need to be a master, just a member of all groups.

If the old owners are causing trouble you can either use idmap or touch and submit (easier to do on Unix). I would leave them alone unless there are problems.

David Smith

I have successfully updated the idmap several times. We have people come and go all the time, leaving behind their orphaned ID as a file owner. It works pretty well - it's just tedious if you have a lot of users on the system to go through and clean the whole thing up. Here's a link to the KB article on how to do it, although it looks like you may already have the instructions: https://customers.autonomy.com/support/secure/solution.jsf?id=71476

P.S. You have to update it for all your stores to get everything, including the workflow store.

HurricanesFan

Thanks Nipper & Smitty.

Don't think links to the old KB articles work anymore, but I think I have enough to go on.

Currently we aren't live on TS750 yet and have an old content store in place, so I think I will try iwidmap as I've not run it before and see how I feel about it.

But when we do go live I will probably go for the conservative option and let the system heal thyself over time with submit.cfg.

Cheers

David Smith

HurricanesFan wrote:
Thanks Nipper & Smitty.

Don't think links to the old KB articles work anymore, but I think I have enough to go on.

I tested the link before I posted it and it works for me. It's useful as somehow I get a lot of "no user" entries that I reset to our admin user. Anyway, do what makes sense for you.

HurricanesFan

I tested the link before I posted it and it works for me. It's useful as somehow I get a lot of "no user" entries that I reset to our admin user. Anyway, do what makes sense for you.

Found my old pw and got the article, thanks, will read tomorrow. So you use this to set to your admin user - are you using a generic login or just a 'placeholder' user?

We're currently debating here as to whether we should have a generic master account or stick with named users - security and all that.

David Smith

We have an admin account but no one logs in as that account - it's the account we run all of our externaltasks as owner of, but all individual users have their own accounts. If not, we'd never know who was doing anything when something needed to be accounted for. When an individual user leaves and their account is essentially orphaned, everything they owned appears in the UI as their old ID from the idmap. So every once in a while, I will go in and clean it up - assigning the orphaned IDs to our admin users UID. That way the UI looks cleaner.

Adam Stoller

HurricanesFan wrote:

(A)
In our TeamSite workareas we have many files/dirs owned by obsolete users [...] and I am convinced this slows TeamSite in Linux due to issues with Linux user lookups of unknown users (anyone else think this?).
(B)
I have decided to change us from many individually-owned workareas to one group-owned workarea, so I can use submit.cfg to gradually change ownerships to generic tsmaster:nlmweb, but I note that although that works well for files it doesn't necessarily change directory ownerships. Note that we use straight Unix groups not "TeamSite groups".

[...]
(C)
I was also looking at running iwidmap, but am edgy at running mass-changes on the content store unless there is a desperate need to do so. Besides my test attempt to run for just 1 user failed with:
./iwidmap -u -c 3013 tsmaster /usr/nlm/iwdata/iw-store/default
Unable to open TeamSite group file /usr/nlm/iw/iw-home/TeamSite/conf/tsgroups.xml
...assume tsgroups.xml is for "TeamSite Groups", which we don't use so why is it needed for this?

[...]

[A] I would not be so quick to assume that lookup of obsolete user information is slowing TeamSite down without gathering more data. I'm not entirely sure how to caputure that data (I'm sure someone here can provide ideas), but generally OS routines for looking up user information from UIDs should be reasonably quick to return either the name or UID if there is no name. That being said, it is generally a good idea to take care of cleaning up the system (TeamSite) to remove references to obsolete user information (more on this in a bit wrt iwidmap)
[B] There are, IMO, few good reasons for having individually owned workareas and, conversely, many good reasons to use shared workareas (including the fact that SitePublisher only works [as I recall] with a single workarea per branch) - so I agree that this is a good change. Directories do not get versioned after creation and thus, once submitted, you will not be able to change their properties (ownr, group, etc.) except via iwidmap.
[C] The reason iwidmap needs to access the tsgroups.xml file is that user entries are listed in the groups file, so if you're mapping a user from "person-A" to "person-B", TeamSite wants to make sure that all references to "person-A" in the tsgroups.xml file are also changed to "person-B". NOTE [RE: A]: iwidmap does not remove or replace the UID (or GID) associated with files and directories - it maps the UID (or GID) to another UID (or GID) such that the file will appear and behave as if it is owned / shared by that other user / group. In this respect, TeamSite would still be seeing files owned by the obsolete user's UID but would display the name of the mapped UID instead - so I'm not sure if that would really provide performance enhancements. NOTE [RE: B]: If you remap the owner of a directory, all files and directories associated with the original owner will be remapped - i.e.: you aren't changing the owning UID of the directory just changing how that UID is mapped across the entire store.

I'm not sure why Nipper doesn't think that files should be owned by individuals (I would agree with not having branches or workareas owned by individuals) but that may just be a matter of taste - what I would do, however - especially because you are on a Unix system, is set things out so that all directories [including the top-level workarea directory] are set with the set-gid-bit enabled (chmod g+s) to ensure that the group-owner of the directory is used as the group-owner for all contents of that directory, and I would also use a TeamSite Group to be that owner (though you could do it with an OS group if you prefer)

HurricanesFan

Thanks Ghoti & Smitty,

this is good stuff for me to digest.

About user lookups, when we had a lot of slowness HP-Aut Engineering advised analysis of pstacks of iwersver.linux process taken at slowdowns indicated that users are not found in the local cache and the server has to perform an OS lookup, which appears to be stalling and causing a delay. In particular I saw a lot of getpwuid()'s, which Linux uses to get the user.

nipper

ghoti wrote:
I'm not sure why Nipper doesn't think that files should be owned by individuals (I would agree with not having branches or workareas owned by individuals) but that may just be a matter of taste - what I would do, however - especially because you are on a Unix system, is set things out so that all directories [including the top-level workarea directory] are set with the set-gid-bit enabled (chmod g+s) to ensure that the group-owner of the directory is used as the group-owner for all contents of that directory, and I would also use a TeamSite Group to be that owner (though you could do it with an OS group if you prefer)

Individuals leave the company. tsadmin doesn't. Not a huge problem but users often complain when the see the owner as S-123-4567-8-91011

The bigger issue is that if the owner is still an individual, then there is a good chance that the group and/or permissions are wrong because submit.cfg is not being used or is incorrect.

Adam Stoller

HurricanesFan wrote:

About user lookups, when we had a lot of slowness HP-Aut Engineering advised analysis of pstacks of iwersver.linux process taken at slowdowns indicated that users are not found in the local cache and the server has to perform an OS lookup, which appears to be stalling and causing a delay. In particular I saw a lot of getpwuid()'s, which Linux uses to get the user.

I would think that getpwuid calls would be made regardless - to map UIDs to userid's - however it could certainly be the case that unmapped, no-longer-existing UIDs could require more processing than either existing UIDs or mapped no-longer-existing UIDs -- the best evidence will be what you see if/when you do the mapping and perform the tests again (please update the thread if/when you do so).

Adam Stoller

nipper wrote:
ghoti wrote:
I'm not sure why Nipper doesn't think that files should be owned by individuals [...]
Individuals leave the company. tsadmin doesn't. Not a huge problem but users often complain when the see the owner as S-123-4567-8-91011

The bigger issue is that if the owner is still an individual, then there is a good chance that the group and/or permissions are wrong because submit.cfg is not being used or is incorrect.

Seeing the SIDs in the UI is, to me, simply an indication that iwidmap needs to be run. I tend to see more complaints that user's don't see their name associated with a file (or most recently modified-by) - more often this is on Windows where they see "SYSTEM", but the same could be said for Unix if all the files show up as owned / modified by "tsadmin" (or similar)

Considering it as an indicator that the submit.cfg isn't working is, I suppose, a reasonable assumption but I don't think the two are so fully linked; you could have UID/SID values show up on a system where the submit.cfg file is working properly and you could have no visible UID/SID values showing on a system where the submit.cfg file is not configured / configured properly.

I think it is valid to suggest that one should configure - and verify - the submit.cfg file, but I'm not sure I'd link it directly with regard to this particular thread's subject matter.

HurricanesFan

Continuing on with this one...

I have an issue with versions, and a possible solution I'd like to run past you because I have to be careful.

We used to have double-branch promotion system, and now I've had to amalgamate the 2 branches consisting of largely the same files into 1. So I've put the first old 'Dev' branch into a new 'common' workarea, and the old 'promotion' branch into the Staging area.

That means of course that the versions often don't match for files. A file in the workarea may be newer by timestamp than it's equivalent in Staging, but the one in Staging may have a later version number. So that means it won't show as modified in the workarea (the system sees it as modified in Staging).

So I need to change that, by writing a script, so that files newer by timestamp in the workarea show as modified. At the same time I might as well fix file owner & group, by perl chown which uses numeric uid's and gid's, and I'll use chown to 'touch' files so they appear modified.

So:

- for those with the same timestamp in the workarea & staging, but different versions, I would do a chown uid:gid and then submit with overwrite to get the versions the same

- but for those newer in the workarea but an older version than in Staging, I would also chown uid:gid but NOT submit. I found this means they will show as modified in the UI for the users (f+ f+ with iwcmp) and the version in the workarea will show as +, BUT it will still have a lower version number than in Staging. I think that is the best I can do, as I don't think there is a way to artificially increment the version number just in the workarea.

Can anyone think of a better approach? Specifically:

- is there a way to artificially increment the version number just in the workarea, so make it "[Staging vers no]+"?

- is it a good idea to chown uid:gid files (rather than use iwidmap, which doesn't have them shown as modified)?

Tricky one to explain, and I'm not sure if anyone else would have had this particular problem.

Cheers

Adam Stoller

I would suggest the following - pretty much regardless of which version is newer:

For each file... compare the content in workarea against the content in staging

If the content is the same do a GetLatest with overwrite on that file to make sure that the file in the workarea is officially in sync with staging.
- If you want to chown/chgrp the file too - do so in the workarea and then submit with overwrite.
If the content is different in the workarea, move the file aside (e.g.: mv foo.html bar), do a GetLatest on that file, and then move the file back in place over the sync'd file (e.g.: mv bar foo.html)
- If you want to chown/chgrp the file too - do so in the workarea, but (as you already indicated) do not submit the file at this point in time - that can happen later once you are sure that the changes are desired.

The point is that the process in #2 above will guarantee that the version in the workarea is assigned the same version as that in staging, with a '+' afterwards - which is, I believe, what you were looking for - yes?