How Big: Repository Size

In our environment we have two TeamSite servers. Right now the repositories have the following statistics:
TS1: 14 G made up of 1.4 million files
TS2: 22 G made up of 2.2 million files

It takes over 24 hours to copy the TS1 data from one server to another via our network (on a weekend). And TS2 has gotten to a point where a copy will fail.

Questions:
Q1. Do other companies have repositories equal to the size and file make-up such as what I listed above?

Q2. If so, what methods do you use to move a repository?

Environment:
TeamSite 6.5
Windows 2003
Backups: Net Apps Snap Images and NDMP off-line backups
(It took a while to get useful backups until the above architecture)

Find more posts tagged with

Comments

msrinivas

Did you run iwfsshrink (refer to CLT manual for more info) to clean up things?

Please remember that doing this might results in slower performance on the boxes and it starts cleaning your content store.

Adam Stoller

In our environment we have two TeamSite servers. Right now the repositories have the following statistics:
TS1: 14 G made up of 1.4 million files
TS2: 22 G made up of 2.2 million files

It takes over 24 hours to copy the TS1 data from one server to another via our network (on a weekend). And TS2 has gotten to a point where a copy will fail.

Questions:
Q1. Do other companies have repositories equal to the size and file make-up such as what I listed above?

Q2. If so, what methods do you use to move a repository?

Environment:
TeamSite 6.5
Windows 2003
Backups: Net Apps Snap Images and NDMP off-line backups
(It took a while to get useful backups until the above architecture)

Some questions come to mind:

)

jed

Have I just entered into a **** contest about the size of my repository?? Smiley Happy

We copy from prod down to dev and test about once/quarter. Our largest store is approx. 110gigs and our transfer rate is anywhere from 6-15 gigs/hour depending on the target server and network activity.

I have found that there are a lot of really slow ways to copy a backing store. Nfs shares, rsync and opendeploy all produced very slow results. Generally we get the best throughput using a command similar to this:

From the production server...
cd /some/backing-store
tar cf - . | dd bs=64k | rsh teamsite-dev "cd /new/target/backingstore; dd bs=64k | tar xfp -

You are on windows so you cannot replicate the above command exactly, but there may be an equivilent. Either way, you need to get a bit creative.

Note: We do a full transfer since we write the production store to a new directory on the test/dev server. If we did an update of the existing store on our dev/test box, we might benefit from using rsync since it can do bit comparisons and only transfer modified files.

dtheimer

Thanks for the replies. I was looking for one of my old post to update it and realized that there were a few new posts.

msrinivas - It has been a long time since a shrink was run. I was not expecting much from the process, but figured it about time to run the utility. Results were a little bit better then I expected. Each of the two stores reduced in size by about 1 to 2G and a few hundred files. So It was a good suggestion/reminder, but not enough change to solve the issue.

Ghoti: Answers to your questions:
Some questions come to mind:
Was TS1's store frozen or the server shut down before you attempted the copy (or, if using a "snapshot" was the server frozen or shut down before the snapshot was taken)?
*** Yes, a process shuts down all the TeamSite services before the NetApps Snapdrive starts the backup.
Was TS2's server shut down before you attempted the copy?
*** Not really -- The copy was from SnapDrive restored image, which if I recall have no pointer back to the actual data and is an independent version.
How are you currently performing the copy?
*** Copy was a Windows "copy - right mouse click" and "paste" over the company network.
Is TS2's store being changed independently of TS1's store?
*** Yes the TS1 & TS2 stores are independent of each other and reside on separate Filers (File servers) in separate datacenters.

Jed - Have you ever looked at how many files make up your 110G of data?

I think TeamSite is pushing the limits on the way a piece of content is broken down in to multiple file and stored on the file system. I think it takes much more time to copy lots of little files then it take to copy the same size of data in one large, but equal file.

I am going to test out a utility called LiveState. I am told by the Windows Admins that if this application can not copy it nothing can...we will see.

msrinivas

I just had a file copy done of a 100GB+ store for migration @ 2GB/hr. This was in Solaris and over the network. I am not sure what kind of disks you have and network etc. but 22GB failing is not good. I usually back up/copy my laptop of 30GB+ much easily than that though at a slower pace.

HTH

Adam Stoller

Ghoti: Answers to your questions:
Some questions come to mind:
Was TS1's store frozen or the server shut down before you attempted the copy (or, if using a "snapshot" was the server frozen or shut down before the snapshot was taken)?
*** Yes, a process shuts down all the TeamSite services before the NetApps Snapdrive starts the backup.

Okay - so the source server is shut down properly before the backup is created - good.

Was TS2's server shut down before you attempted the copy?
*** Not really -- The copy was from SnapDrive restored image, which if I recall have no pointer back to the actual data and is an independent version.

TS2 is the target server - right? You're saying that you did not shut down the target server before trying to copy the TS1 backing store onto TS2? and you're copying the data to the actual location of the backing store that TS2 is using? - that would be "bad" and likely to cause backing store corruption.

How are you currently performing the copy?
*** Copy was a Windows "copy - right mouse click" and "paste" over the company network.

This is probably not a very efficient copy mechanism. Also - I do not know if copy will replace the entire contents of the target store with the source store - or if it's going to try to do individual file updates and additions. If the latter - this is "bad" and likely to result in backing store corruption. Backing stores do not "merge" at the file system level.

Is TS2's store being changed independently of TS1's store?
*** Yes the TS1 & TS2 stores are independent of each other and reside on separate Filers (File servers) in separate datacenters.

Again - if you're trying to copy a store (e.g. "/default") from TS1 to TS2 - and TS2 already has a store by that name - and the contents of the stores on both servers are changed independently of one another - you should not use a copy mechanism that will try to "merge" changes from the source files into the target files - you need to replace the entire contents of the target store with the entire contents of the source store.

If I've misinterpreted what you're trying to do (i.e. it's not "iw-store/default" => "iw-store/default") please clarify.

If I've interpreted what you are trying to do correctly - I'd be extremely surprised if Interwoven would support your TS2 server.

dtheimer

TS1 & TS2 are production servers. The need to copy/move content (repository) would be to re-fresh a development or QA environment or during a hardware upgrade of a server. The repository is copied to the Dev/QA server over the existing content or one respority/store is brought down and other (new) one is activated using the Interwoven documented procedure.

We (WCM team, Windows Admins, Backup team) think the issue is a derivative of the way a file is broken down into many small files when TeamSite adds the file to the repository. This is addition to the number of branches (150 per TS) and versions being created by the business Users.

Potential solutions are: LiveState software, Using a NetApps SnapDrive solution and Firewire Hard Drive.

Adam Stoller

TS1 & TS2 are production servers. The need to copy/move content (repository) would be to re-fresh a development or QA environment or during a hardware upgrade of a server. The repository is copied to the Dev/QA server over the existing content or one respority/store is brought down and other (new) one is activated using the Interwoven documented procedure.

We (WCM team, Windows Admins, Backup team) think the issue is a derivative of the way a file is broken down into many small files when TeamSite adds the file to the repository. This is addition to the number of branches (150 per TS) and versions being created by the business Users.

Potential solutions are: LiveState software, Using a NetApps SnapDrive solution and Firewire Hard Drive.

I'm still a bit unsure of what you're doing - is your intention is to completely replace the backing store on TS2 with the one from TS1 - or to "bring over changes" from TS1 to TS2 even though TS2 may have additional changes that don't exist on TS1?

The latter is, I believe, just asking for headaches (and perhaps an ulcer).

The former should not be that much of a problem assuming:

provide for this kind of functionality.If you're already using NetApps - I think it would be worthwhile to check into that first - it's been a while since I've worked with their boxes, but I seem to recall that they tended to do a very good job at things like backups and recoveries - and this would essentially be just that - a backup from TS1 and a recovery to TS2 - if so, it's likely to be far more efficient than the other methods.With respect to the number of files storead within the backing store - I don't believe the actual content files are chunked up across multiple data files (I could be wrong though) - I believe that each version of each file is stored as an individual file in the backing store. The name and location of the files, however, is definitely a proprietary algorithm. The "little" files tend to be related to metadata (file attributes, EAs, etc.) and the number of these in a 6.x system should be less than what existed in 5.x and significantly less than what existing in 4.x and earlier versions of TS.The fact that there are so many files should not be that big an issue unless you are forcibly copying each and every file across the network. This is where something like OpenDeploy and rdist tend to shine - they can perform comparisons across the network without actually shipping the data back and forth (e.g. timestamps, file size information, checksums) and only ship the data for files which actually need to be transferred. Windows Copy is, as far as I know, a "dumb" program (fine for expressly copying a few files from point A to point B, but woefully inadequate for what you're trying to do).