Import 10000 Documents

Hello, we have a new solutions with LES 10.5. We are going to put in 10000 existing invoices. The invoices should be ordered by category. What is the fastest way to import them to LES? Would be nice if i get some suggestions.

Regards,

André Freiburghaus

data world consulting ag

Find more posts tagged with

Comments

John_Henderson

Hi Andre,

We have done large imports via both web services and using the inbuilt Object Importer. For Object Importer, the control files where created either using the OpenText migration engine as part of a larger project, but also since then we have also written some bespoke C# code to loop through a file directory and create the relevant xml object importer xml control files. Object Importer is faster in terms of loading to Content Server, but of course you also have the time taken to create the XML Object Importer control files. I think from memory we limited each control file to have a maximum of 2000 documents in each one (so you would need to run 5 imports etc).

We have also used the C# .Net / CWS web services to upload documents, it's usually slower, but gives more flexibility in terms of applying permissions, categories and dynamic folder locations. We use these a lot for smaller document uploads, and have at times used them for data import type tasks.

Object Importer would be my first choice, the xml structure to generate is like the below (if not used before)

<?xml version="1.0" encoding="UTF-8"?>

<node type="document" action="create"><location>Enterprise:Folder here </location>

<file>\\workingfiledirhere\123.doc</file>

<createdby>Admin</createdby>

<attribute name="attribute1here">att1 value</attribute>

<attribute name=" attribute2here ">att 2 value</attribute>

</category>

</import>

…next doc

Hope that helps.

John_Henderson

Should have also said, Object Importer is perhaps best suited to one of larger migrations, however developing an application using web services / CWS upload may be more useful if you want to continue to automatically import invoices / documents in smaller numbers over time etc.

Jean-Francois Larente

We use ObjectImporter a lot for creating complex directory structures. We’ve actually also come across a 3^rd party tool from Altova called MapForce. MapForce allows you to create detailed and complex mappings from/to various sources. In our case I use a database query as the input and generate a control.xsd file which includes categories, security clearance supplemental markings, shortcuts, nickname, etc.

I’ve changed the actual values to fake for privacy reasons, but here is a snippet of the xml for a folder generated using MapForce. I created a folder structure with 600 items and variable sub-items using our custom tables.

<location>Enterprise:Sample:Sample2:Sample3</location>

<title>FolderName</title>

<owner>Admin</owner>

<ownergroup>OT-Enterprise-CM</ownergroup>

<attribute name="SampleAttr1"> SampleAttr1Value</attribute>

<attribute name=" SampleAttr3"> SampleAttr3Value</attribute>

</category>

<attribute name="Document Group">Guide</attribute>

<attribute name="Document Type">Administration Guide</attribute>

</category>

<suppmarkings>Guides</suppmarkings>

</securityclearance>

</node>

JEAN-FRANÇOIS LARENTE
Senior Systems Analyst
Conversant Intellectual Property Management T613-599-9539, 1402 M 613-415-6852 F 613-591-8148
W conversantip.com

Conversant Intellectual Property Management Inc.

NOTICE: The contents of this email and any attachments to it contain confidential and/or legally privileged information from Conversant Intellectual Property Management. The contents of this email are only for the use of the intended recipient. If you are not the intended recipient, you are hereby notified that any disclosure, copying, printing, distribution, or the taking of any action in reliance on the contents of this email is strictly prohibited. If you have received this email in error, please notify us by email immediately.

From: eLink Entry: Content Web Services Forum [mailto:otdncontentwebservices971forum@elinkkc.opentext.com]
Sent: January-20-15 4:47 AM
To: eLink Recipient
Subject: Import 10000 Documents

Import 10000 Documents

Posted byjohn.henderson@iow.gov.uk (Henderson, John) On 01/20/2015 04:45 AM

Hi Andre,

Object Importer would be my first choice, the xml structure to generate is like the below (if not used before)

<?xml version="1.0" encoding="UTF-8"?>

<node type="document" action="create"><location>Enterprise:Folder here </location>

<file>\\workingfiledirhere\123.doc</file>

<createdby>Admin</createdby>

<attribute name="attribute1here">att1 value</attribute>

<attribute name=" attribute2here ">att 2 value</attribute>

</category>

</import>

…next doc

Hope that helps.

[To post a comment, use the normal reply function]
Topic:	Import 10000 Documents
Forum:	Content Web Services Forum
Content Server:	Knowledge Center

jdonpaul

If you want to develop an application, then you will want to look at the DocumentManagement and Authentication web services. The first thing you want to do is authenticate so that you can recieve a token to use against the Content Server that you are making a connection to. Once you have recieved a token, you can then use that token in each subsequent call you make to Content Server using the DocumentManangement web service. A good example of how to authenticate and upload a document would be the LLTreeView found here:

https://knowledge.opentext.com/knowledge/cs.dll/Open/ContentWebServices

I hope this gets you started in the right direction.

jdonpaul

CS 10.5 documentation for web services on OTDN has a special “Uploading Documents” example in the “Using CWS” section as well. The example can also be found here:

https://knowledge.opentext.com/knowledge/cs.dll?func=ll&objId=45500683&objAction=browse&sort=name&viewType=1

eLink User

If you have OI and a programmer who understands how to create flat files it is very easy.I usually rig up something fast with a access database or excel VBA.

Here is one such file it is not very user friendly but I had used this to create blank folders for somebody.I tried it today and it did work like I wrote it several years back.
https://knowledge.opentext.com/knowledge/cs.dll?func=ll&objId=17381257&objAction=download&viewType=1

OI is a nicely written framework so I was able to use it for things that OT did not include like email and stuff like that.I like the implementation.

CWS is ok until you hit the category data structure which is not for the faint.The examples OT put in this forum build a category object

and then create the object setting the metadata which is not how livelink people work.In many companies a contract programmer does not have any category creation privileges so in my organization to train such people I was able to make some headway into some C# and Java samples which I shared on my blog.Not prod quality...
http://livelink.in/2014/04/16/so-you-want-to-change-categoriesattributes-of-a-object-in-livelink-via-cws/

..Appu

I wonder how one could debug livelink if Fiddler, Wireshark and Softerra LDAP browser was not invented and not free

On Tue, Jan 20, 2015 at 11:41 AM, eLink Entry: Content Web Services Forum <otdncontentwebservices971forum@elinkkc.opentext.com> wrote:

Import 10000 Documents Posted byjdonpaul@opentext.com (Don-Paul, Jerome)On 01/20/2015 12:38 PM

CS 10.5 documentation for web services on OTDN has a special “Uploading Documents” example in the “Using CWS” section as well. The example can also be found here:https://knowledge.opentext.com/knowledge/cs.dll?func=ll&objId=45500683&objAction=browse&sort=name&viewType=1
[To post a comment, use the normal reply function]Topic:Import 10000 DocumentsForum:Content Web Services ForumContent Server:Knowledge Center