nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

7.2.1 Linux - error while deploying to LSCS

Adam Stoller

I tried to search for relevant threads, saw a couple, but nothing seemed to really pinpoint a resolution, so ....

Environment[indent]TeamSite, OpenDeploy, LSCS, LSDS 7.2.1 on Linux with WebSphere Application Server[/indent]
Problem[indent]Attempting deployments to production environment via modified copy of PLC workflow - failing with generic error
[indent]

LSCS StartDeployment: Error while deploying to LSCS. Deployment Status: Failed Please look OD/LSCS logs for more details

[/indent][/indent]
A little background[indent]

Find more posts tagged with

Comments

Bowker

Did you look in the filelist to see if that had/has a really long filename in it?

/usr/Interwoven/LiveSiteCSAuth/authoring/tmp/20f6e73b9115e7fe6f5314c4539af03d/filelist/lscs.filelist

nipper

On the receiver look at the lscs.log maybe the idol log as well

Is there an lscs license ?

The import is a catch all error.

I have see it for a few things. One is that if there is an EA called Title and title, that will cause this error.

nipper

Also, I assume you changed the error to look like this:

http://TARGET_SERVER:15060/lscs/v1/import

???????

Check lscs-conf.properties on the LSCS server

Adam Stoller

Did you look in the filelist to see if that had/has a really long filename in it?

/usr/Interwoven/LiveSiteCSAuth/authoring/tmp/20f6e73b9115e7fe6f5314c4539af03d/filelist/lscs.filelist

The longest entry in the file list is 147 characters long

Adam Stoller

On the receiver look at the lscs.log maybe the idol log as well

Is there an lscs license ?

The import is a catch all error.

I have see it for a few things. One is that if there is an EA called Title and title, that will cause this error.

In what path would I find the "lscs.log" (and "lscs license"?) - on which server (source or target)

nipper

Locate to use locate

locate lscs.log (on Linux)

On the LSCS server

Adam Stoller

Also, I assume you changed the error to look like this:

http://TARGET_SERVER:15060/lscs/v1/import

???????

Check lscs-conf.properties on the LSCS server

Results:[html]

Method not supported on this uri: GET /lscs/v1/import

[/html]

The lscs.log on /usr/Interwoven/LiveSiteCSRT/log on the Receiver is 0-bytes?

I found the properties file (or at least one instance of it) am I looking for anything in particular there?

Adam Stoller

Locate to use locate

locate lscs.log (on Linux)

On the LSCS server

Heh - I can't run locate right now - I don't have 'root' and my sudo access is restricted and doesn't include access to /var/lib/mlocate/mlocate.db

Adam Stoller

I'm getting concerns from my customer that when we cleared out /usr/LSCSRT-Store that we might have removed something that was needed.

Can anyone confirm for me that this is just a root directory for temporary file deployments for use for loading LSCS - and that there's nothing there that anything should be dependent upon?

Adam Stoller

Another piece of information - not sure if it's related:[indent]If we look at
http://TARGET_SERVER:15060/lscs/v1/rulesengine/execute?project=//PROD/default/main/ABC/DEF/GHI
we get back:[indent]
XML Parsing Error: no element found
Location: http://TARGET_SERVER:15060/lscs/v1/rulesengine/execute?project=//PROD/default/main/ABC/DEF/GHI
Line Number 1, Column 1:[/indent][/indent]Any idea what I should be looking at for this?

nipper

[oh303d@www-lst-01 ~]$ locate lscs.log
/usr/Interwoven/LiveSiteCSRT/log/lscs.log
[oh303d@www-lst-01 ~]$ locate lscs-conf.properties
/usr/Interwoven/LiveSiteCSRT/runtime/webapps/lscs/WEB-INF/classes/lscs-conf.properties

Deleting /usr/LSCSRT-Store on the LSCS server was not a good plan. There is information store as AFAIK. You may need to reset the project.

what does the debug panel show

http://TARGET_SERVER:15060/lscs/admin/debugpanel/

On my LSCS server all the tabs work.

nipper

Heh - I can't run locate right now - I don't have 'root' and my sudo access is restricted and doesn't include access to /var/lib/mlocate/mlocate.db

Talk to your sys admin, I do not have read access to /var/lib/mlocate either but I can use locate with no trouble

I bet that the binary is not setuid:

ls -l /usr/bin/locate
-rwx--s--x 1 root slocate 28184 Mar 17 2009 /usr/bin/locate

Andy

Adam Stoller

[oh303d@www-lst-01 ~]$ locate lscs.log
/usr/Interwoven/LiveSiteCSRT/log/lscs.log
[oh303d@www-lst-01 ~]$ locate lscs-conf.properties
/usr/Interwoven/LiveSiteCSRT/runtime/webapps/lscs/WEB-INF/classes/lscs-conf.properties

Deleting /usr/LSCSRT-Store on the LSCS server was not a good plan. There is information store as AFAIK. You may need to reset the project.

Well, apparently the removal of the LSCSRT-Store was not a good idea, fortunately we did it in a way we could easily restore it - and that, along with restarting all of the WebSphere apps seems to have fixed the issue - for now.

I'm worried about the LSCSRT-Store - it's got 25,000 sub-directories in it - which means that performance is just going to continue to degrade and ultimately probably fail because there are too many entries in the directory. I'll probably follow this up with Support.

what does the debug panel show

http://TARGET_SERVER:15060/lscs/admin/debugpanel/

On my LSCS server all the tabs work.

Right now - it shows:[html]

[/html] - not sure what "all the tabs work" means?

Adam Stoller

Talk to your sys admin, I do not have read access to /var/lib/mlocate either but I can use locate with no trouble

I bet that the binary is not setuid:

ls -l /usr/bin/locate
-rwx--s--x 1 root slocate 28184 Mar 17 2009 /usr/bin/locate

Andy

Bingo - whether or not they'll change it remains to be seen ;-) Thanks

nipper

Right now - it shows:[html]

[/html] - not sure what "all the tabs work" means?

I meant the other buttons, Diagnostics, Document Query, etc.

If you hear anything from support, please post.

HKN1104

Ghoti

Can you please post commit-plc.xml in conf/lscs.

Thanks

Adam Stoller

I meant the other buttons, Diagnostics, Document Query, etc.

If you hear anything from support, please post.

Not on site right now, but I don't recall seeing any buttons when viewing that URI.

As for Support - I raised the issue of the large number of subdirectories within LSCSRT-Store and was essentially told "don't remove them"

The content of LSCSRT-Store are the actual raw content files + the lscsidx for IDOL for that deployment + the manifest file for that deployment. The actual content is what takes the bulk of your space so the only way to reduce the size would be to delete the content but if you want to keep your system's integrity that is not desirable. That's why when you tar'd the directory and removed it your LSCS system would have stopped retrieving content.

So, if this content is what has been pushed out from your TS to be available on your LSCS and you are going to be keeping on deploying content in the future then you need to look into increasing the size of this directory as this is the amount of space that is absolutely required to maintain LSCS.

In addition LSCS itself maintains it's own recycling mechanism. Any file that you deploy to LSCS is maintained twice (The most recent copy deployed + the last copy). So, if the same file is deployed the third term then the original copy is deleted from the LSCSRT-Store directory. That's why you may see 20,000 directories but some of the old directories will only have the lscsidx and the manifest file and not the actual content because that has been replaced by a newer version of that file in a most recent deployment. This cycle repeats itself. At anytime no more than two copy of a file exist. This is implemented by product design to make sure roll-back of deployments is possible when there is an issue with LSCS during deployment.

We do not implement any archiving mechanism because this data needs to be available right away when queried. Zipping and unzipping will cause a delay in a query to be processed that will slow down your web page loads.

So, in conclusion this directory needs to be untouched for the product to work as expected. You should look into increasing the size of this directory and increase it taking into account the number of deployments in the future.

I've continued the dialog with them because I still feel that this is a fundamental flaw in the design that could easily (perhaps) have been averted by utilizing a directory structure (based on year, year/month, or something along those lines) in which the likelihood of reaching 32,000 entries in a single directory (the potential "magic number") would be greatly diminished. We'll see.

In the meantime, I did a bit of googling and found that it appears, at least for Linux, that this may not be as big an issue as it used to be - though whether LSCS and OpenDeploy use the mechanisms that Linux provides is another story.

Rick Poulin

As for Support - I raised the issue of the large number of subdirectories within LSCSRT-Store and was essentially told "don't remove them"I've continued the dialog with them because I still feel that this is a fundamental flaw in the design that could easily (perhaps) have been averted by utilizing a directory structure (based on year, year/month, or something along those lines) in which the likelihood of reaching 32,000 entries in a single directory (the potential "magic number") would be greatly diminished. We'll see.

I only skimmed through the thread, so apologies if I'm stating something that's already been said... but they did exactly this in 7.3.1. The immediate children folders of 'LSCSRT-Store' are now just the first three hex digits (therefore allowing only 16^3=4096 combinations). Subfolders of those have the same naming convention as before (long context ID) but at least they're grouped in a semi-reasonable fashion.

Adam Stoller

I only skimmed through the thread, so apologies if I'm stating something that's already been said... but they did exactly this in 7.3.1. The immediate children folders of 'LSCSRT-Store' are now just the first three hex digits (therefore allowing only 16^3=4096 combinations). Subfolders of those have the same naming convention as before (long context ID) but at least they're grouped in a semi-reasonable fashion.

That's good to know (you'd think the Support guy would indicate as much - especially when my last question was: "Has this been improved in 7.3.1 or 7.3.2?")

However, my customer is on 7.2.1 and likely to be so for at least the next 4+ months, so we'll just deal with clearing out the log directory but not the LSCSRT-Store directory ;-)

mart2001

I would also look at the Websphere log which might give you some more clues.

[QUOTE=ghoti]I tried to search for relevant threads, saw a couple, but nothing seemed to really pinpoint a resolution, so ....

Environment[indent]TeamSite, OpenDeploy, LSCS, LSDS 7.2.1 on Linux with WebSphere Application Server[/indent]
Problem[indent]Attempting deployments to production environment via modified copy of PLC workflow - failing with generic error
[indent]

LSCS StartDeployment: Error while deploying to LSCS. Deployment Status: Failed Please look OD/LSCS logs for more details

[/indent][/indent]
A little background[indent]

Adam Stoller

I would also look at the Websphere log which might give you some more clues.

Did you really have to quote the entire original message for that?

As indicated, there was a disk-space problem and then we needed to restart the services - and then we needed to restore the LSCSRT-Store, which I hadn't realized was needed - and then restart the services again.

The number of directory entries directly within LSCSRT-Store is apparently fixed in the 7.3x version of LiveSite - but we're not in a position to upgrade any time soon - so we'll just have to keep a watch on it and possibly do a major re-set of the store at some point (a non-trivial task outlined in a KB)

At this point I'm going to just close the thread as I don't think there's anything more of value to put into it.