nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

fastest possible OD config ?

nipper

Has anyone tried to find the fasted possible OD config, when you are blasting a lot of data ? I am getting about 1 GB per hour on a pretty good connection. I am going to turn off transactional & date comparison (since it is a new deployment)

Any other ideas ?

Andy

Find more posts tagged with

Comments

Adam Stoller

Fastest deployment would be a sourceFilesystem with filelist deployment as there is no comparison performed beforehand.

Next fastest would be a sourceTeamsite deployment, because the comparison takes place locally and then the assets are deployed.

Least fastest would be sourceFilesystem without filelist, because this involves a comparison across the network.

That's the "high-level" view.

Turning on encryption will reduce the speed of each of the above.

Making them transactional should not greatly change the overall time of deployment, but will increase the amount of disk space required.

Enabling various other features (ACL setting or preservation type options, user mapping, doDeletes, etc.) will also increase the amount of time for the overall deployment.

So the absolute fastest deployment would probably be sourceFilesystem with filelist without ACL settings without doDeletes, etc.

However - fastest is not always best or "right" - the "slowest" method (sourceFilesystem without filelist) tends to be the basis for the "best" method when wanting to make sure that assets deleted on the source side are cleaned up from the target side.

It's pretty much relative to the "needs".

Did that answer your question - or did I just go way into left field here?

--fish
(Interwoven Senior Technical Consultant)

Migrateduser

Data compression (introduced in OD 5.6) may also speed things up. Depends on whether the savings in transfer time more than offsets the time added for the compression/decompression steps.

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

swong74

follow-up to Andy's question :

A transactional deployment will push out only content that have changed (new files set aside for a moment) correct? But it writes it as .iwnew and should there be 100 files on the deployment, 100 .iwnew all at once? When is .iwold created ... only when file already exists?

My observation had been that whether or not I have transactional on or off, if I set date-different=yes, I see .iwnew and .iwold on the receiver side and for all files that I have changed timestamps on. So, if 10GB worth of content had been accidentally timestamp changed, a deployment could potentially eat up 30GB assuming all that 10GB is under one deployment?

- sw

Migrateduser

Only one set of files gets deployed. For transactional deployments, the sequence of steps are pretty much the following, given a file "foo":

Make a copy of the original file: foo.iwold
Transfer the new file: foo.iwnew
Rename the new file to the original name: foo.iwnew -> foo
Remove the old copy (i.e., foo.iwold)

So you need 3x the disk space on the target during the transactional deployment (foo, foo.iwold, foo.iwnew).

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

Migrateduser

And if you are doing non-transactional, you need 2x because foo will get deployed as foo.iwtmp and then get renamed. This is because OD does not update a file in place. If it did, you would risk corrupting a file if the deployment terminated abnormally.

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

nipper

I assume Todd that it does this in a batch mode, so if I have
a 10 GB deployment (and say permissions adjusted every file) then I will use 20 GB for the short tem.

or does it deploy everything in this dir & then move ?

Andy

Migrateduser

Deployment is done in phases. So the transfer phase would generate all the new files, then the commit phase is where all the new files are flipped to the new name.

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

Adam Stoller

To clarify - since we've been talking about both transactional and non-transactional deployments.

In transactional - the steps (as preivously described) happen on a deployment-wide basis

In non-transactional - the process happens on a file-by-file basis.

--fish
(Interwoven Senior Technical Consultant)

ProfessorX

Will Compression for OD 5.6 work for reverse deployment as well?

The Professor-

Duplicata.ssd.zip

Duplicata.pdf

Migrateduser

It should work, since the transfer rules should apply in both forward and reverse deployments.

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

raoliver

To repeat and number the steps before I ask my question:

1. Make a copy of the original file: foo.iwold
2. Transfer the new file: foo.iwnew
3. Rename the new file to the original name: foo.iwnew -> foo
4. Remove the old copy (i.e., foo.iwold)

Question: If I implement a DNR script (transactional deployment) will it
1) Run inbetween steps 3 and 4 above?
2) If the DNR fails and signals the failure to Open Deploy, will Step 4 above not run and also copy the old file (i.e., foo.iwold) back to the original name?

Migrateduser

1) There is no DNR trigger point between transactional commit and cleanup (i.e., between steps 3 and 4)

2) Behavior of a transactional deployment when a -2 response code is returned by the DNR script is outlined in the OD 5.6 SP1 Release Notes, p. 51. The behavior depends upon which trigger point is being used.

Todd Scallan
Group Product Manager
Interwoven
t: 408-530-7167
e: tscallan@interwoven.com

Migrateduser

I found this topic interesting as I'm currently trying to find the best OD config for our system. Currently the system is set up to deploy every branch hourly base on directory comparision. This config slows down our sever. The justification of its original designer was if we have hundreds of workflow, realtime filelist deployment for each workflow would consume a lot of resources.

My question is, what's the threshhold in terms of number/size of files for a filelist deployment before we see performance problem?

Adam Stoller

If you're using OD 5.6 SP1 (or later) - switch to using:
iwodcmd start configname ...
instead of:
iwodstart configname ...

iwodstart initiates a new JVM for each invokation, iwodcmd start does not do so - making it far more efficient on the Base Server side.

Coming in OD 6 (not sure if it's been released yet?) - the Receiver can coordinate multiple deployments to the same target area (I believe) making it far less likely to run into problems with multiple deployments for the same source/target area stepping on one another.

In general, filelist deployments tend to be short and quick on both sides of the transfer and the only thing you need to content with as part of your workflow deployments is to make sure each invokation uses a unique '-inst instname' parameter to avoid trying to fire off the exact same deployment as one which is already running.

This can usually be coupled with a full-branch comparison deployment occuring once per night (or even less frequently depending on your needs) to act as a source/target sychronizer "just-in-case" (the main advantage being that it can also take care of doDeletes to remove obsolete files from the target side -- assuming you have a reasonable mapping of source area / target area contents)

This is what we do at my current customer's site and the only problem we've run into was the size of all the logfiles on the Receiver side - for which we wrote our own clean-up script that runs nightly to remove all OD logs older than 3 days (we have a similar script on the Base Server side that actually compresses all logs first, and then removes all compressed logs older than a certain threshhold - and that's for all logs - not just OD)

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

Migrateduser

Thanks! This is good to know.