How to increase the rate of image insertion in documentum content server

Hi Friends

I am using documentum API code to insert tiff images and its metadata into documentum content server. The rate of insertion is only 4-5 images/minute. Please let me know if there is any way to increase the number of images to insert into documentum. I need atleast 12-15 images to be inserted into documentum otherwise it will lead to a big problem. We are using documentum content server 6.5sp1 version. Please help.

Find more posts tagged with

Documentum

Comments

aflowers001

Run a DFC trace to see if you can identify any bottlenecks, also look at you code and see if it can be made multi threaded/multi process.

Are the image files large? If so is it network latency that is an issue, the trace will also help here.

Could be worth you posting sample code or pseudo code to show the algorithm you are using.

DCTM_Guru

Where are your images stored? If its on shared drive and you can map that drive to the Content Server, running your custom import app on the Content Server will eliminate any network latency.

pettitk

We wrote a multi-threaded java program that imports documents. Try doing something like that. Any single threaded app will be just that… Are the images large? Sounds odd to take that long. Maybe you have some hardware issues or limitations going on…

Kyle

dctm_dev

Thread is not helping much in this case.

Image files are of 18-50 KB. I dont think its too large.

Sample code or pseudo code is pasted below to show the algorithm we are using.

IDfSysObject loOldObject = (IDfSysObject)loSession.getObjectByQualification("cust_type where <condition>");
    if (loOldObject == null)
    {
        IDfSysObject loObject = (IDfSysObject)loSession.newObject("cust_type");
        loObject.setString("attrvalue", attrname);
        ........
        ........
        loObject.setContentType("tiff");
        if (File.Exists(imagepath))
        {
            File.AppendAllText("C:\\LoggerPath\\Export_Dctm_Log" + DateTime.Now.Day.ToString() + DateTime.Now.Month.ToString() + DateTime.Now.Year.ToString() + ".txt", "Image path--" + imagepath + Environment.NewLine);
        }
        loObject.setFile(imagepath);
        String loLinkPath = "/application/DOCUMENT";
        loObject.link(loLinkPath);
        loObject.save();
        img_objid_qry = _m_DfClientX.getQuery();
        img_objid_qry.setDQL("select r_content_size from cust_type where <condition>");
        imgobjid_coll = img_objid_qry.execute(loSession, (int)tagDfQueryTypes.DF_READ_QUERY);
        while (imgobjid_coll.next())
        {
            File.AppendAllText("C:\\LoggerPath\\Export_Dctm_Log" + DateTime.Now.Day.ToString() + DateTime.Now.Month.ToString() + DateTime.Now.Year.ToString() + ".txt", "size" + imgobjid_coll.getDouble("r_content_size") + Environment.NewLine);

        }
        //Newly appended code snippet Up
        flag = "Success in main image insertion";
    }

dctm_dev

Images are stored in a shared drive and response is ok like 31 - 80 ms.

Content Server is a unix box.

dctm_dev

Thread is not helping much in this scenario.

Content server is a unix box. Network is responding like 31-80 ms.

lastnitescurry

Suggest you do a DFC trace as Andy has recommended, see here for howto

https://community.emc.com/thread/136852

Multi threaded would be good but only after you figure out why single threaded is so slow.

Assuming you have fixed/eliminated the recommendations of Andy, Johnny and Kyle then what happens to your objects on save? Can you disable these actions?

TBO code?
Image transformation or thumbnail creation
?
?

btw... your code sample:

What is this "File.AppendAllText". A logger? Why roll your own, use DfLogger. Much simpler and has timing information included.
After the save, do a fetch and you should be able to get the r_content_size with out a query.
you never close your collection (imgobjid_coll) which will result in brutal performance!!!

dctm_dev

There is no TBO on save. We are using the File.AppendAllText as this is done in visual studio (dotnet). The collection is made closed but i didnt post such as I only posted a part of my main code.

Thanks for suggesting fetch concept. I will implement the same.