Question about massive data injection

During my next project the client boast quite a few documents that will need to be moved from their custom ecm to documentum..

quite a few, is around 20-22 millions of files growing with a rate of around 1500/3000 daily...

Is there any tool for massive batch insert of those documents into the content repository?

Because all things considered a migration of that size seems to me it would take... quite a lot of time... (single-thread, one file at a times puts me off an esteem of an huge amount of days)

At the moment i'm thinking about doing some sort of multi-threaded parallel import via dfc..

Is there some more apt way to deal with migration of these size? first time I'm handling a migration of this magnitude and I'm not so sure if there's any methodology to follow in this case.

Thanks in advance,

Vito

Find more posts tagged with

Note:

If there are Accepted Answers, those will be shown by default. You can switch to 'All Replies' by selecting the tab below.

Accepted answers

bacham2

A multi-threaded DFC program would do the job. Try to run it on the Content Server machine, with the content files accessible locally, to avoid transfer time. You may want to look at High-Volume Server as well. Not sure if it meets your requirements (there are several restrictions about what you can do with documents under HVS).

Apart from that, some EMC partners provide tools for mass import.

All comments

bacham2

Apart from that, some EMC partners provide tools for mass import.

witchedwiz

yeah i was under the impression that a custimized component for webtop leveraging the dfc would do the trick..

regarding hvs.. honestly i considered it, but since i have zero field-knowledge with it I preferred going the old-way..

out of curiosity, what limitations are we talking about to be exact?

bacham2

No versioning for example. I suggest you read the HVS manual to get a better idea.

witchedwiz

mmm..

yeah that definitely would be an issue when migrating documents...

oh well, I guess the dfc-boy within me will have some hard work to do for optimizing pure java code for such mammut-sized migration.

thanks for clarifying the whole situation, much appreciated