nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Extracting Documents from LiveLink using LAPI

Dotty_Civita

Hi,I have a requirement to write a code in java to extract all the documents with the folder structure from livelink server if a project number is given. Currently i have a code which will walk through 1 level at a time. i.e. if a project number is given it will fetch the folder first then check if document is there inside the folder, if so then fetch that document and so on.Is there a way by which I give the project# and all the documents are pulled with the folder structure.Any pointers would be helpful.Thanks, Rikesh

Find more posts tagged with

Comments

Appu_Nair

A couple of thingsA)Are you trying to find all objects(documents) within a project object(subtype=202) or are you saying that the "project number" is the dataid of a regular folder.B)I would probably code with a ListObjects call with subtype=144 that will give you all the objects of documents.C)If a real Project subtype occurs once has to multiply the negative dataid by -1 and then do the ListObjects ask project have a negative concept.Here is a link to how a travesral can be obtained on a livelink tree including projects if you wanted to see how it is done.Regardless of what approach if there is a nesting of folders/containers one has to walk the tree,unless of course you can use oracle connect by which will do the walk for youhttp://communities.opentext.com/communities/llisapi.dll/open/7762536

Dotty_Civita

hi Appu,Thanks for the quick response.Project number Id is the dataId of a regular folder. I'm curerntly using the ListObjects call with subtype=144 to fetch documents inside one particular folder. But if the folder has subfolders then the subfolders and all its contents will not be fetched. Yeah as you had mentioned,painful traversing through each folders to get the document is what I'm currently doing.The problem I'm facing is I have around 20k projects and data above 200Gig which will take its own sweet time to extract . Would it be possible for you to tell me how much time is taken for downloading a proj typically .Rikesh.

Appu_Nair

Do you have Object Expoter in this install.That uses oscript so might be a tad faster.Also I used SQLqueries to accomplish the listobjects call when I had to do a dump of a huge ll syystem to disk and I had to get all versions as well.I posted that in Greg Giffiths web sitehttp://greggriffiths.org/livelink/development/lapi/bulkfetch/The SQL libraries I used are deprecated now but you can undesrtand the logic,with each dtree dataid I ask dversdata for all versions and burn to disk obvioulsy I had to get folders also.Also I designed so that many high level folders could be done.I did no try to dumep starting at enterprise node id.I may have a stable evrsion with xml files and setup info.Write to me and I will try to find that if you want to take that approach.Also if you want to try oscript it has better tree walkers and stuff called nodecrawler that will be more apt for this.

Bhupinder_Singh

Message from Bhupinder Singh <bsingh@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Also see this article:

Why does ListObjects not work for a Project?

https://knowledge.opentext.com/go/3499257

- Bhupinder

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Tuesday, December 02, 2008 12:01 PM
To: eLink Recipient
Subject: Extracting Documents from LiveLink using LAPI

Extracting Documents from LiveLink using LAPI		Posted by cvsadmin (Civita, Dotty) on 12/02/2008 11:56 AM
In reply to: A couple of things		Posted by anair@alitek.com (Nair, Appu) on 12/02/2008 11:40 AM

hi Appu,

Thanks for the quick response.

Project number Id is the dataId of a regular folder.

I'm curerntly using the ListObjects call with subtype=144 to fetch documents inside one particular folder. But if the folder has subfolders then the subfolders and all its contents will not be fetched. Yeah as you had mentioned,painful traversing through each folders to get the document is what I'm currently doing.

The problem I'm facing is I have around 20k projects and data above 200Gig which will take its own sweet time to extract .
Would it be possible for you to tell me how much time is taken for downloading a proj typically .

Rikesh.

Appu_Nair

Forget what I said about ObjectExporterI beleive that gives you an xml file only another livelink understands so are useful in migrations only.What was I thinking:)Write code so that only manageable folders are processedAugment more than one lapi program to run(don't go crazy on this as lapi also has to get in the queue for threads)Maybe create standby livelink instances to process requests excusively for the lapi traffic.Run summary timings logs and check if LL is thread starved.Those are all things that I think could preceivably speed up the exporter

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

You can do this with a few lines of code using EasyLAPI:

------------------------------

public void export(LLObject node, String filespec) throws EasyLAPIException {
File myFile = new File(filespec + File.separator + node.getName());

System.out.println("Exporting: " + myFile.getAbsolutePath());

    if (node.isContainer()) {
        myFile.mkdirs();
        for (LLObject child : node.getChildren()) {

            export( child, myFile.getAbsolutePath() );
        }
    } else if (node.isDocument()) {
        node.getDocumentVersion(myFile);
    }
}

------------------------------

EasyLAPI makes the Project structure transparent. In other words, you don't need to worry about the associated volume (that negative DataID stuff) when traversing or adding content to Projects.

I hope this helps.

Chris

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Tuesday, December 02, 2008 17:20
To: eLink Recipient
Subject: Extracting Documents from LiveLink using LAPI

Extracting Documents from LiveLink using LAPI

Posted by cvsadmin (Civita, Dotty) on 12/02/2008 11:14 AM

Hi,

I have a requirement to write a code in java to extract all the documents with the folder structure from livelink server if a project number is given. Currently i have a code which will walk through 1 level at a time. i.e. if a project number is given it will fetch the folder first then check if document is there inside the folder, if so then fetch that document and so on.

Is there a way by which I give the project# and all the documents are pulled with the folder structure.

Any pointers would be helpful.
Thanks, Rikesh

Appu_Nair

That to me looks awesome.Wow

Dotty_Civita

I was able to extract documents from a sample set of projects - 6656 documents from 55 projects totalling a size of 5.25 GB took 3hr 13 minThis was using 2 threads running simultaneously. Something intresting I noted here was when I used asingle thread for extraction it took almost double the time but when I use 3 threads, the time saving is just 10 mins or so. And if I take the extraction stats of each project- the individual extraction time increases as I increse the number of threads.Any pointers? does the livelink server have a restriction?

Appu_Nair

What is the lapi user account connected.Does it have SA on its privs that would basically not do any permissions checking and i would think will be faster.Have you tried Christopher Meyer's EasyLAPI method Is that faster?As far as a single instance of livelink is considered it is usually configured to run 5 threads and usually you can run more than one livelink instance on a server until the databases starts bottleneckingYou are averaging 0.5 document per second Is my math right6656/193 mins*60I beleive I did get upload speeds of 1 doc/sec using OI sadly I don't know if file sizes make a difference.

Dotty_Civita

I'm gettign an exception while trying to get a connection:Testing connection to livelink-dev .... com.opentext.api.LLHTTPClientException: Problems with Web Transport () at com.opentext.api.LLConnect.readResponseHeaders(LLConnect.java:1085) at com.opentext.api.LLConnect.executeHTTP(LLConnect.java:1378) at com.opentext.api.LLConnect.execute(LLConnect.java:334) at EasyLAPI.EasyLLSession.execute(EasyLLSession.java:74) at EasyLAPI.EasyLLSession.execute(EasyLLSession.java:47) at com.opentext.api.LAPI_DOCUMENTS.GetServerInfo(LAPI_DOCUMENTS.java:1090) at EasyLAPI.Session.getServerInfo(Session.java:1003) at EasyLAPI.Session.getCharacterEncoding(Session.java:1019) at EasyLAPI.Session.isUTF8(Session.java:1031) at EasyLAPI.Session.setCharacterEncoding(Session.java:987) at EasyLAPI.Session.testConnection(Session.java:1188) at EasyLAPI.Session.testConnection(Session.java:1169) at rik.EasyLAPI.main(EasyLAPI.java:27)The code used:public class EasyLAPI { // Change these three variables to match your instance private static final String FolderName = "20000"; private static final String DocumentName = "This is my new document"; private static final String filepath = "C:/temp/file.txt"; public static void main(String[] args) { try { // First thing is to create a Session or connection to a // Livelink instance //SessionProperties sp = new SessionProperties(args[0]); //ession s = sp.getSession("myServer"); Session s = new Session("livelink-dev", 3099,"livelink", "username", "password"); // check the connection s.testConnection(); // create the folder and add the document to the new folder LLObject newFolder = s.getEnterpriseWorkspace().addGetFolder(FolderName); newFolder.addDocument(DocumentName, filepath); } catch (java.lang.ArrayIndexOutOfBoundsException e) { System.out.println("Usage: java AddFolderAndDocument servers.ini"); } catch (Throwable e) { e.printStackTrace(); } }}Am I missing anything?

Claudia_Meyer

Hi Dotty, Try one of the other constructors. The one you're using: public Session(String, int, String, String, String) is for tunneling through a proxy server. Try changing it to this: new Session("livelink-dev", 3099, "username", "password");I hope this helps.Chris

Dotty_Civita

Chris this is the error that I got when I used the above code...Testing connection to livelink-dev .... com.opentext.api.LLUnknownFieldException: LLValue unknown field name: characterEncoding at com.opentext.api.LLNameTable.get(LLNameTable.java:42) at com.opentext.api.LLAssoc.get(LLAssoc.java:69) at com.opentext.api.LLValue.toInteger(LLValue.java:809) at EasyLAPI.Session.getCharacterEncoding(Session.java:1019) at EasyLAPI.Session.isUTF8(Session.java:1031) at EasyLAPI.Session.setCharacterEncoding(Session.java:987) at EasyLAPI.Session.testConnection(Session.java:1188)Thanks,Rikesh

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Hi Rikesh,

What version of Livelink are you using? The Session.testConnection() method has a dual purpose:

test if an authenticated request can go through; and
to set the character encoding.

It's possible older versions of Livelink don't return the character encoding information on the LAPI.GetServerInfo() call, therefore causing the error. I will fix this for a subsequent release, but for now just comment out the testConnection line.

Let me know how that goes.

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Thursday, December 04, 2008 16:24
To: eLink Recipient
Subject: LLValue unknown field name characterEncoding

LLValue unknown field name characterEncoding		Posted by cvsadmin (Civita, Dotty) on 12/04/2008 10:22 AM
In reply to: Hi Dotty,		Posted by cmeyer (Meyer, Christopher) on 12/04/2008 04:54 AM

Chris this is the error that I got when I used the above code...
Testing connection to livelink-dev .... com.opentext.api.LLUnknownFieldException: LLValue unknown field name: characterEncoding
@ @ at com.opentext.api.LLNameTable.get(LLNameTable.java:42)
@ @ at com.opentext.api.LLAssoc.get(LLAssoc.java:69)
@ @ at com.opentext.api.LLValue.toInteger(LLValue.java:809)
@ @ at EasyLAPI.Session.getCharacterEncoding(Session.java:1019)
@ @ at EasyLAPI.Session.isUTF8(Session.java:1031)
@ @ at EasyLAPI.Session.setCharacterEncoding(Session.java:987)
@ @ at EasyLAPI.Session.testConnection(Session.java:1188)

Thanks,
Rikesh

Dotty_Civita

Thanks Chris....this worked...and this is really wow..really appreciate your help.

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Great! Let me know if you have any other questions or problems.

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Thursday, December 04, 2008 17:31
To: eLink Recipient
Subject: That worked!!!!

That worked!!!!		Posted by cvsadmin (Civita, Dotty) on 12/04/2008 11:28 AM
In reply to: RE LLValue unknown field name characterEncoding		Posted by cmeyer (Meyer, Christopher) on 12/04/2008 10:55 AM

Thanks Chris....this worked...and this is really wow..really appreciate your help.

Dotty_Civita

hi Chris,I Was able to extract the documents but only the last version of the document is being extracted. Is there any way to get all the version of the document?

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Hi Dotty,

This is possible, but there is a small bug with EasyLAPI v0.5 that prevents this from working. Grab v0.5.1 from the download area, which has the fix.

Here's the code snippet to get all versions.

------------------

int maxVersionNum = node.getVersionNum();

for (int v = 1; v <= maxVersionNum; v++) {
    try {
        node.setVersionNum(v).getDocumentVersion("C:/filenameV" + v + ".txt");
    } catch (com.opentext.api.LLIllegalOperationException e) {
        System.out.println("Version " + v + " likely doesn't exist.");
    }
}

------------------

The getVersionNum() method returns the latest version number unless it was explicitly set by the setVersionNum() method. A LLIllegalOperationException exception occurs if you try to download a version that doesn't exist.

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Thursday, December 04, 2008 18:49
To: eLink Recipient
Subject: multiple versions of file

multiple versions of file		Posted by cvsadmin (Civita, Dotty) on 12/04/2008 12:46 PM
In reply to: RE That worked!!!!		Posted by cmeyer (Meyer, Christopher) on 12/04/2008 11:56 AM

hi Chris,

I Was able to extract the documents but only the last version of the document is being extracted. Is there any way to get all the version of the document?

Dotty_Civita

hi Chris,this seems to fetch the version..but however there is a small problem.I have a file with name testDocument and it has two versions1.jpg and 2.jpg The file that is fetched by the above method will fetch documents as testDocumentv1 and testdocumentv2. we lose the file name and the file extensions.I was trying to use the doc.getSession().getDocuments().ListVersions(doc.getVolumeID(), doc.getParentID(),value) to list the versions...value is LLValue.in the above method I got LLvalue as ? all the time. This however worked with the LAPI ..so thought it would work here. unfortunately it is not.

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Hi Dotty,

The getName() method returns the name you see when browsing the folder containing the document. The example I gave with the filename is just an example to demonstrate the concept. You could modify it like this:

node.setVersionNum(v).getDocumentVersion("V" + v + " " + node.getName());

If you need to append the extension (if it's not part of the filename already) you can write a function to do a lookup against the mimetype. The mimetype can be retrieved using the node.getMimeType() method, which fetches the mimetype for whatever version you have set with the setVersionNum() method.

EasyLAPI doesn't have a method yet to retrieve the filename for a particular version, but I'll add that in the next release.

The problem with your code snippet is the doc.ParentID() parameter. That's likely the DataID of the parent folder, which doesn't have any version information.

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Friday, December 05, 2008 15:51
To: eLink Recipient
Subject: multiple versions

multiple versions		Posted by cvsadmin (Civita, Dotty) on 12/05/2008 09:46 AM
In reply to: RE multiple versions of file		Posted by cmeyer (Meyer, Christopher) on 12/05/2008 04:58 AM

hi Chris,

this seems to fetch the version..but however there is a small problem.

I have a file with name testDocument and it has two versions
1.jpg and 2.jpg

The file that is fetched by the above method will fetch documents as testDocumentv1 and testdocumentv2. we lose the file name and the file extensions.

I was trying to use the
doc.getSession().getDocuments().ListVersions(doc.getVolumeID(), doc.getParentID(),value) to list the versions...value is LLValue.

in the above method I got LLvalue as ? all the time. This however worked with the LAPI ..so thought it would work here. unfortunately it is not.

Dotty_Civita

hi Chris,actually when I gave doc.dataId instead of doc.parentID it worked. Thanks man...thanks for all the help.This is the code I used:if (node.getSession().getDocuments().ListVersions(node.getVolumeID(), node.getDataID(), value) == 0) {for (int i = 0; i < value.size(); i++) { node.getDocumentVersion(myFile + "_" + value.toString(i, "FILENAME"));System.out.println(value.toString("COMMENT")); } }Thanks,Rikesh

Dotty_Civita

Hi Chris,Is there any mehtod by which I can fetch the VALINT value from LLATTRDATA table using Easy LAPI?Thanks,Rikesh

Claudia_Meyer

Message from Christopher Meyer <cmeyer@opentext.com> via eLink

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">eLink

Hi Rikesh,

You wish to fetch an attribute value for a given category? You can do something like this:

LLCategoryVersion catVersion = node.getCategoryVersion(catName);
int value = catVersion.getAttributeLLValue(attrName).toInteger();

Just replace catName and attrName with the name of the category and attribute.

You can also add the indexValue parameter if it's multi-valued:

int value = catVersion.getAttributeLLValue(attrName, indexValue).toInteger();

If fetching a string you can simply use:

String value = catVersion.getAttributeValue(attrName);

chris

From: eLink Discussion: LAPI Discussion [mailto:lapi@elinkkc.opentext.com]
Sent: Tuesday, December 09, 2008 16:04
To: eLink Recipient
Subject: VALINT value from LLATTRDATA table

VALINT value from LLATTRDATA table		Posted by cvsadmin (Civita, Dotty) on 12/09/2008 10:00 AM
In reply to: Extracting Documents from LiveLink using LAPI		Posted by cvsadmin (Civita, Dotty) on 12/02/2008 11:14 AM

Hi Chris,

Is there any mehtod by which I can fetch the VALINT value from LLATTRDATA table using Easy LAPI?

Thanks,
Rikesh

Dotty_Civita

Thanks Chris... that helpedThanks,Rikesh

stacktrace.txt