nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Store-related question

Corey_Null

Background: I just joined the team that manages our TeamSite implementation. This implementation has been in place several years, and we're currently on version 6.1 in production. A teammate is in the process of upgrading to 6.5, and we're scheduled to upgrade to 6.7.1 by the end of the year. I've also just completed the 6.7 System Admin class a month or so ago.

Issue: One of the biggest problems that I see with the current environment is that we have a single, massive store. It's over 100Gb in size, with hundreds of branches and hundreds of thousands (maybe millions) of files. We can't run utilities like iwfsck or iwfsshrink on it as they take weeks to run. The store is severly corrupt.

Once we get to 6.5, I would like to carve the store into multiple stores. The way I understand it, we need to figure out how many files each of the branches contain. Then we need to carve the store up accordingly, so that we can have several stores with a fairly equal amount of files in varying branches. First of all, is this assumption the best way to carve up stores? Second, is there an easy way to determine the number of files in each branch? A perl script that I could run against teamsite or the IFS mount maybe?

Next, when I go to add new stores, is there a best practice naming standard? I think that I read somewhere that we can have a max of 8 active stores per server. Also, we figured out during the upgrade to 6.5 that the iwmigrate doesn't migrate workareas. Does someone have a way to migrate workareas and files (with extended attributes) after iwmigrate is complete?

I'm pretty sure that this is going to be a massive undertaking, and there are several good ways to accomplish it. Any feedback would be greatly appreciated. Thank you in advance.

Find more posts tagged with

Comments

nipper

You are asking a lot of questions that are hard to answer in a sound bite. What someone says may *sound* good, but there is a lot more detail that you need.

The good news is that you are asking the correct questions, so you are not coming in from far left field.

1st off, consider bringing in someone who has done this before, even just 3 or 4 weeks of a top notch architect will be a big help. Talk to your IW sales critter for details about brining in someone from IW. There are also independents out there.

Concerning store size, splitting that up into multiple stores is an excellent idea. Did you buy the extra stores ? Yes 8 is the max, naming is not a big deal.

Per branch size, you can write a script to figure out how much each branch takes. You would need to, for each file, run iwrlog and get every version of each file, and then add the size of each. Not trivial to write, but will take longer to run than it will to write.

The real question is not how big is each branch I have, but how big each new branch (and thus store) will be ? When you split stores using iwmigrate it is common to only take a subset of the history.

This is a start.

HTH

Andy

Adam Stoller

While it's good that you included the version information in your post you didn't include the platform information - and some of the questions asked may have different answers depending on the platform.

FWIW - even if iwfsshrink takes a long time to run - you really should run it. You can start it and then pause it during the day if you find that it's adding too much load - or is it failing because you don't have enough disk space to handle freeing up disk space (non-intuitive problem - but it sometimes comes into play if copies of things are being made prior to removal of other things)

As Nipper asked - did you actually purchase licenses for the stores? Although you can create them without having the license - you will be in violation of your Interwoven license if you have more than one store active (for more than temporary purposes) without having purchased the license (and then I think you need to generate a new license on the support site to avoid it telling you that you are in violation).

If you think that you're just dealing with years of bloat and you don't really need all the history - using a temporary second store to iwmigrate the last N editions of content from each branch and then removing the branch from the default store and migrating them back - is a reasonable solution -- it will still take time and you will probably have to declare certain branches as being out-of-service until the process is complete - but it will allow you to shrink the size of the store and continue to use one store at the end.

The workarea issue is separate - there are scripts out there on the support site and/or perhaps on this site (either in forums or in the tech articles?) for copying all the EAs on files (preferably just modified files) and you can tie that together with a tarball or some such to preserve the actual modified data - and then untar / re-create the EAs afterwards (or in the target store) -- alternatively, if you're using a recent version of OD (I believe any version of 6.x will do) you can enable its feature for transferring EAs and create a deployment config file to transfer the files and their EAs from a branch in one store to a workarea in another store (and perhaps back again if you're going with the one-store-final-result approach)

I don't think there's any specific naming conventions for stores - though you should probably keep the names "short-and-sweet" and avoid punctuation characters ($!? etc.)

Hope that helps

Corey_Null

Thanks for the replies. Lots of great information.

Unfortunately, we're currently on Windoze 2003 Standard server with quad physical 2.7Ghz Xeons (with hyperthreading enabled for a total of 8 procs) and 3.5Gb of RAM. I think that we're upgrading to new hardware and Win2k3 Advanced Server during or after the 6.5 upgrade, but definitely before the 6.7.1 upgrade later in the year.

I'll look into running the iwfsshrink. I didn't know you could pause it. Will RTFM on that. We really need to also implement an archival solution as we're a financial/health insurance company and can't really delete anything. At this point, I doubt that there's much whitespace.

Licensing isn't an issue.

nipper

BTW, officially 2003 Standard Edition is not supported, you are *supposed* to use Enterprise Edition.

Now, that being said many people use SE successfully. It is just not officially supported.

Corey_Null

I thought that Standard was supported with at least 6.1. I just checked the release notes, and they don't specify. They just state Windows 2000 (SP4) and Windows 2003.

I know that the documentation states that Enterprise (I mistakenly said Advanced before) is required for 6.5 and above. I'm sure that, since that's the case, we'll be upgrading the OS before we upgrade to 6.5.

Corey_Null

Ok. We have a new server, running the correct OS, and TeamSite 6.5.
We have the old store mounted and created a new store along with it. We ran iwmigrate to migrate one branch from the old store to the new store. We then ran OD to copy workareas and files from the old store to the new store. Unfortunately, in the new store, some of the file versions are missing (after running the iwmigrate of course). Any known reasons why that would occur?

13adam13

You had mentioned possible backing store corruption earlier, perhaps that is the culpit.
I have seen where store corruption will result in the display of files that are un-readable.

Also, did you output your iwmigrate into a log file? Have you looked through there for evidence of failures?

Adam Stoller

Ok. We have a new server, running the correct OS, and TeamSite 6.5.
We have the old store mounted and created a new store along with it. We ran iwmigrate to migrate one branch from the old store to the new store. We then ran OD to copy workareas and files from the old store to the new store. Unfortunately, in the new store, some of the file versions are missing (after running the iwmigrate of course). Any known reasons why that would occur?

Are these "old" versions that are missing - or "new" versions?

"old" versions missing could be, as mentioned, due to backing store corruption issues - or because you didn't migrate all the older editions.

"new" versions missing is usually because you didn't created editions on all of your branches prior to doing the migration - or didn't migrate all the latest editions.

Corey_Null

Are these "old" versions that are missing - or "new" versions?

"old" versions missing could be, as mentioned, due to backing store corruption issues - or because you didn't migrate all the older editions.

"new" versions missing is usually because you didn't created editions on all of your branches prior to doing the migration - or didn't migrate all the latest editions.

At this point, I'm talking about "new" versions. There are certainly "old" versions missing (due to the corruption issue I mentioned above), and I expect those to be missing in the new store as well. However, in certain branches (it isn't across the board), there are versions of files in the old store, but not in the new store. In other words, in the old store, if I look at a file there might be 2 versions, but when I look at the same file in the new store, the versions are at 0+.

CRB

are you saying that, for example:

when you go to ed_005 in the old store you look at a file A and you see that it has 2 versions
and when you go to ed_005 in the new migrated store you look at a file A and you DONT see that it has 2 versions? if so then there is a problem...

but it will most likely be what Ghoti said, you didnt publish and edition of that file before migrating... to verify it, try the example i explained above, go to the same edition in both stores and check versions ona file.

Corey_Null

are you saying that, for example:

when you go to ed_005 in the old store you look at a file A and you see that it has 2 versions
and when you go to ed_005 in the new migrated store you look at a file A and you DONT see that it has 2 versions? if so then there is a problem...

This is exactly what I am saying. Editions have been published. Based on this, and what I know about versioning in stores (
http://devnet.interwoven.com/forums_vb/showthread.php?t=13921&forumid=11 ), I think it might be related to the known corruption in the "old" store.