nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Running multiple data deployments transactionally

System

I'm sure there's a way to do this but I'm at a loss as to how to proceed...

I need to run several datadeployments transactionally. They all occupy the same config file but are invoked by different "iwdd"'s. Since these definitions cannot be combined into the same dbschema, how do I go about running several or many datadeployments transactionally?

Thanks,

Dave

Find more posts tagged with

Comments

Bill Klish

Clarification question: Do you need to run several of these iwdds and then have a single commit at the end, or can each of these be committed at once?

If you need to have them all be done in a single large transaction, run iwdd1, iwdd2, iwdd3, commit, I don't think that is possible.

DD executes a commit (or rollback) on your connection at the end of each deployment.

That being said, you would have to get creative and perhaps could take a stored procedure approach, but I assume you still need to get all the data loaded, so that might not help much either.

Migrateduser

Yeah, your answer sounds right and I was hoping otherwise but I guess this isn't possible. I searched through the posts and found a 3-year old message that looked hopeful:

http://devnet.interwoven.com/forums_vb/showthread.php?t=2695&forumid=12&highlight=multiple+transactional

Unfortunately, it's vague at the end.

Adam Stoller

Yeah, your answer sounds right and I was hoping otherwise but I guess this isn't possible. I searched through the posts and found a 3-year old message that looked hopeful:

http://devnet.interwoven.com/forums_vb/showthread.php?t=2695&forumid=12&highlight=multiple+transactional

Unfortunately, it's vague at the end.

It's been a while since I've played with DD (thankfully ;-) - but I believe you can create a definition that execs other definitions (or something like that) such that you can call the "wrapper" definition and have it all run transactionally (though this may not work if multiple "sub" definitions need to refer to data deployed to tables in other "sub" definitions).

Migrateduser

Well, fish, you managed to get me dizzy.

I don't think it's quite what you're envisioning. I've actually already done a pretty sweet workaround to this but I think this topic bears some discussion.

I'd like to run DD using dbschema rather than a do-sql block. I need to run 6 data deployments transactionally -- so, if deployment #5 fails, 1 through 4 roll back and #6 is never attempted. Actually, they don't need to be sequential but you get the idea.

Not that this adds any more complexity but I also need to run this with OpenDeploy so I had it kicking off one DNR wrapper script that I thought to invoke all of the DataDeploys... but that's pretty much where I left it when I became confident that my approach was not going to work.

Dave

Adam Stoller

Well, fish, you managed to get me dizzy.

I don't think it's quite what you're envisioning. I've actually already done a pretty sweet workaround to this but I think this topic bears some discussion.

I'd like to run DD using dbschema rather than a do-sql block. I need to run 6 data deployments transactionally -- so, if deployment #5 fails, 1 through 4 roll back and #6 is never attempted. Actually, they don't need to be sequential but you get the idea.

Not that this adds any more complexity but I also need to run this with OpenDeploy so I had it kicking off one DNR wrapper script that I thought to invoke all of the DataDeploys... but that's pretty much where I left it when I became confident that my approach was not going to work.

Dave

What I'm talking about is a single DD config file with multiple definitions (def1, def2, def3, etc.) each of which has its own dbschema, etc. - and an additional definition (def0) which basically looks like this:

<deployment name="def0">
    <exec-deployment use="def1"/>
    <exec-deployment use="def2"/>
    ...
</deployment>

So - instead of running DD multiple times to invoke the individual definitions (def1, def2, ...) you run it once for def0 - and that I believe could then be transactional across all the other definitions.

However, I could be wrong - this is just based on distant memories.

Migrateduser

Ah, I see what you mean then. I'll try that tomorrow and will write back on my findings.

Dave

Bill Klish

I think you may be on the right track. Totally forgot that this is now possible via the dbloader elements, just not sure whether or not the commit is executed at the end of all these (honestly have never tested it).

Basically, you would create your 6 dbscema.xml files and create separate definitions for each of your them using dbloader elements. Then in the overall deployment section you would do it exactly as fish described, similar to a standard file deployment except you are pushing to a DB.

I am curious to know if this will actually work. My guess is that DataDeploy will execute a commit on each of the dbLoader elements, but it would be cool if it didn't until the end.

Let us know how you make out.

Migrateduser

Yep, that worked! And, apparently it does a commit at the end, too. I thought that the multiple exec-deployment was only available for OD.

Bill Klish

Only possible risk to this approach is a database performance issue if you are dealing with a lot of operations before that commit (i.e. loading a ton of data). That would be the only possible pitfall.

Glad it worked.

jed

Yep, that worked! And, apparently it does a commit at the end, too. I thought that the multiple exec-deployment was only available for OD.

What version are you on? Unless you are running od 6.02 SP1 + latest patches or the latest release (6.1?), this may not work for 3+ schemas. Guess who requested that patch Smiley Happy

If you are not on a recent release, OD will zig instead of zag at times.

As a side note, this also works with a standalone deployment as well.

Example attached.

Migrateduser

Jed,

Good question -- I'm running 6.0.1 on Win2k. I've tested it quite a bit in the 3+ schema scenario and it actually works quite well. Of course, I'm not sure I can tell the difference between a "zig" and a "zag". Smiley Happy

Dave

jed

Jed,

Good question -- I'm running 6.0.1 on Win2k. I've tested it quite a bit in the 3+ schema scenario and it actually works quite well. Of course, I'm not sure I can tell the difference between a "zig" and a "zag".

Dave

zig instead of zag == metadata going to the wrong schema. Mabe the bug was isolated to solaris or a specific use case of ours.

I think 6.0.1 is also suseptable to running out of memory during lots of dbLoader usage. Again, this could be isolated to Solaris.

Migrateduser

Maybe -- it might also be the database platform and/or JDBC driver(s). I haven't had memory issues yet but I'm sure I'll post something if I do!

jed

Maybe -- it might also be the database platform and/or JDBC driver(s). I haven't had memory issues yet but I'm sure I'll post something if I do!

Now that I think about it, there were many bugs with dbLoader in the OD 6.01 days. Not sure exactly when all of these were fixed, but I highly recommend getting to 6.02 SP1 + patch 62024 sooner than later (or the latest release). Some of the bugs include:
* could not specify non-cache mount point such as /.iwmnt which cause cache issues (probably moot for windows)
* failures when a metadata field was over X number of characters (~1024)
* failures when a metadata field contained a character that is not XML safe (example: &)

Our use case is metadata --> Oracle. Other use cases such as DCR --> "other DB" may not be affected. Who knows.

Amit_LnT

Yep, that worked! And, apparently it does a commit at the end, too. I thought that the multiple exec-deployment was only available for OD.

Hi Dave,
It will great, if you could explain me how did you mange to work this transactional deployment. I have a similar requirement. But the difference is that, first I want to call a do-sql deployment and then normal db schema deployment.
Here is how my config file is.

<deployment name="Article_Text_multi">
<exec-deployment use="pre_dd"/>
<exec-deployment use="Article_Text"/>
</deployment>
Both deployments indivisually works fine.
Pre_dd is a do-sql deployment. I want to make them transactional.
But when I run the command, it gives me parsing error.
***ERROR - Starting deployment.
Reason from server: DEPLOY_CONFIG_FILE
Details : Deployment config file (d:\iw-home\OpenDeployNG\conf\mhrs_
geo_prod_Article_datadeploy.xml) has XML parsing error: The markup in the docume
nt following the root element must be well-formed.
Is there any way to do this.?

Env: Teamsite 6.5 SP2, windows server 2003

Thanks,
Amit Bhoyar

Amit_LnT

What I'm talking about is a single DD config file with multiple definitions (def1, def2, def3, etc.) each of which has its own dbschema, etc. - and an additional definition (def0) which basically looks like this:
<deployment name="def0">
    <exec-deployment use="def1"/>
    <exec-deployment use="def2"/>
    ...
</deployment>
So - instead of running DD multiple times to invoke the individual definitions (def1, def2, ...) you run it once for def0 - and that I believe could then be transactional across all the other definitions.

However, I could be wrong - this is just based on distant memories.

Hi Ghoti,
I am running through the same problem of making two deployments transactional. The first one is pre_dd deployment and second is schema deployment. Can you give some direction to me ? As the two deployments command parameters differ. what should I give in the third deployment ?
OD . 6.0.1

Adam Stoller

Hi Ghoti,
I am running through the same problem of making two deployments transactional. The first one is pre_dd deployment and second is schema deployment. Can you give some direction to me ? As the two deployments command parameters differ. what should I give in the third deployment ?
OD . 6.0.1

You can pass as many parameters as you want through the command line via the '-k' option - so you can pass parameters specific to each of the [sub] definitions.

That being said, I don't work with DD all that frequently - so someone else will have to assist you in any further design / debugging issues.