Documentum xCP Load Balanced Solution for High Throughput

Good Day/Evening all,

I'm having a serious challenge with Documentum xCP 2.2 on Content Server 7.1. I've built a solution that collects xml files with embedded images from a FTP location extracts images and stores them in the content server. Sounds simple enough but I'm having the following challenges:

1) I need to process 1.5 million records within 24 hours

2) When creating multiple folders against the parent folder I'm getting a CONCURRENT_LINK_ERROR

3) Adding additional Content servers gives me an DM_STORAGE_E_MKDIR_FAILED error. As well as the above mentioned error.

I must get 3.5 million records processed by Monday any assistance would be greatly appreciated.

Joe

Find more posts tagged with

Note:

If there are Accepted Answers, those will be shown by default. You can switch to 'All Replies' by selecting the tab below.

Accepted answers

NiallSynapps

E_CONCURRENT_LINK_FAILURE will happen more often due to a change in how the Content Server handles folder synchronisation. This was introduced in CS 7.1 P02. There is now (by default) an exclusive lock on a folder during an update. That means that (by default) two threads cannot update the same folder at the same time. This is a particular challenge if you are creating large folder structures within a transaction which link to the same parent folder. For example if you had a Cabinet called "Cases" and created a sub folder called "Case Type A" then you cannot create two cases in the "Case Type A" folder at the same time. You can try this yourself in DA. Create a largish folder structure somewhere and try and create two copies at the same time in different sessions in the "Case Type A" folder.

This is a particular issue if you have large transaction contexts where you're creating large folder structures wrapped in transaction all linked to the same parent (which is a very typical case management use case). You can switch of this behaviour with a server.ini flag, but I'm not going to publish this flag as it is not recommended by support due to possible data integrity problems.

The only fix for this using pure xCP (i.e. no Java) is to avoid multiple threads performing any operation on the same folder concurrently. This essentially requires all ingestion operations that need to perform folder updates to be single threaded.

All comments

lastnitescurry


joebruton510 wrote:

1) I need to process 1.5 million records within 24 hours
I must get 3.5 million records processed by Monday any assistance would be greatly appreciated.
Joe

Good luck


3) Adding additional Content servers gives me an DM_STORAGE_E_MKDIR_FAILED error. As well as the above mentioned error.

see:


2) When creating multiple folders against the parent folder I'm getting a CONCURRENT_LINK_ERROR

see: and

Multiple thread applications need to be designed to separate simple tasks (get a document into a repository) from complex tasks (organize where document is to be located, build relationships, ..). My latest importer did 100 documents per second last weekend, which would be 8.6 million per day. My investment in reading "Java Concurrency in Practice" last Summer paid off

joebruton510

Hi Brian

I have looked into your suggestions and although it gives insight into the problem it does really assist with solving my problems


Adding additional Content servers gives me an DM_STORAGE_E_MKDIR_FAILED error. As well as the above mentioned error.

This problem only occurs when running my second content server and then the problem happens on both content servers my thought on the cause of the problem is that both of the content servers are trying to create the folder on the storage during a transaction and when it commits the transaction the other content server has already created the folder. This could be a problem with the way that I have configured the 2nd content server host machine. When I run two content servers on a single host this problem does not occur the problem only occurs when I'm running the 2nd Content Server host.

When creating multiple folders against the parent folder I'm getting a CONCURRENT_LINK_ERROR

These folders are created with xCP as part of a workflow process so although the solution could work if I was to create my own import methods my solution is collecting the files from a FTP site using BPS and then processing them within the xCP workflow. So my conclusion based on the information that you have provided is that this must be a bug on the BPM folder creation method and needs to be corrected by EMC.

NiallSynapps