nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

LS 7.2.1: fan-outs and additional environments

Adam Stoller

Environment[indent]TS/OD/LSCS/LSDS 7.2.1 + WebSPhere on Linux[/indent]
Issue[indent]Trying to figure out what I need to do (and where I need to do it) in order to configure PLC-based deployment tasks to perform [essentially] a fan-out deployment.

I.e.: For our production environment, there will be four web servers and three application servers. I want to be able to have the deployment tasks in the PLC-like workflow be able to deploy all the web / static content (e.g.: images, css, js, etc) to all four of the web servers and all the applicaton data (e.g.: page files, iwov-resources/*) to all three application servers - preferably as fan-out deployments rather than a set of serialized deployments.

In most cases, I'm talking about a distinct set of servers (Web / App) for each environment, but there's at least one case where we expect to have two environments ("Dev" & "Test") share a set of servers. In all cases, "App" refers to LSCS + LSDS + WebSphere running on the same server.

In one ("Dev") environment, there will be only one Web and one App server. In all the other environments there will be two (or more) Web servers and two (or more, likely three) App servers - which while they may be load-balanced are not (as far as I can determine thus far) "clustered" in the "Microsoft Cluster" sense - so they would require a fan-out deploment process to keep them all in sync.[/indent]What I've done so far[indent]
I've looked through the manuals, and while it seems to get touched on, I don't seem to be able to fully grasp the details (surprise).

I tried searching the forums - and come up with the following:[indent]this is kind of a summary of multiple threads - and a bit of a catharsis too. I've ried to make sure to provide reasonable and accurate paths and file names - something that the documentation (and many of the forum posts) fail to do - so that this will be more helpful to others going on...[/indent]

Anyway - I'm planning on trying to pursue this (phased approach, farms first for a single environment, then adding other environments) and we'll see what happens.If anyone has any useful contributions to make with regard to the above (like verying the answers to some of the questions I raised, and/or detailig anything that I left out) - please do so. Otherwise, stand by and I'll try to update this when I actually get something done.

Find more posts tagged with

Comments

kailash1

First here is what I understand:

You have Webservers Load balanced and App servers Load balanced separately.

Your issues are:

1. Why are there multiple deployment configs for LSCS
2. LSCS fan-outs and Replication farm external to deployment config related question
3. LSDS Fan out

Is that correct understanding?

Based on above understanding I have following questions and suggestions:

Do you have OD-HOME/conf/lscs/commit-deploy-plc.xml?

LSCS needs 2 deployments, one for Payload another for commit or rollback. So with 7.2.1 P1 OpendeployWebroot.xml is merged with LSCS commit-deploy-plc.xml. So After Successful deployment of LSCS Pay load commit deployment happens and adding LSDS definition to this make both LSCS and LSDS Transactional.

1. It will be easier if LSDS Share Databases as it’s read-only.
2. I recommend using internal Farms for LSCS and you could use $TargetNode for farm Name. Please refer to ts_72_author_install_rev4_en.pdf page 271 section “Fan-out Deployment”
3. Also you could create different definitions for each env and name them based on your target nodes (DEV, STAGE, PROD) the target node variable to exec deployment name.
4. For LSDS PLC Customizations please refer to ts_72_spuser_en_rev2.pdf Appendix B. Also refer to “LiveSite: Adding another production runtime node” KB https://customers.autonomy.com/support/secure/docs/FAQ/Interwoven/61437.html

Adam Stoller

[...]LSCS needs 2 deployments, one for Payload another for commit or rollback. So with 7.2.1 P1 OpendeployWebroot.xml is merged with LSCS commit-deploy-plc.xml. So After Successful deployment of LSCS Pay load commit deployment happens and adding LSDS definition to this make both LSCS and LSDS Transactional.

Are you saying that in P1 you do not have both OpenDeployWebroot.xml and commit-deploy-plc.xml?

If so - then I'm pretty sure I'm dealing with a non-patched system, and I believe both of those files do exist on my server (I'm pretty sure the latter gets called, I'm not entirely sure about the former - I'll have to check when I get on-site in about an an hour.

I do know that TeamSite is build # 50 - do you (or anyone) know if that represents a patched or non-patched version of 7.2.1?

1. It will be easier if LSDS Share Databases as it’s read-only.

I assume you mean "per-environment" as in "dev", "test", "int", "prod" LSDS servers. Fair enough, but which of the various configuration files controls this?

The documentation is so spread-out and (IMO) chaotically organized, that it's difficult to keep them all straight.

2. I recommend using internal Farms for LSCS and you could use $TargetNode for farm Name. Please refer to ts_72_author_install_rev4_en.pdf page 271 section “Fan-out Deployment”

Thanks for the manual and page reference - I knew I had seen something about that before, but couldn't find it because I was looking in the wrong manuals (sigh)

However, that being said - is the only reason you recommend using internal (as opposed to external) farm declarations because that's how it's documented there - or is there some deeper, technical, reason for doing so? (I think externally defined farms are better as it allows the deployment configuration files to be more easily migrated between "DEV" and "PROD" TeamSite servers)

Also - you are saying I should use $TargetNode for the farm name - however, the documentation you reference still clearly shows it as being used for one of the N nodes in the farm - and not the farm name itself. So again, I have to ask (despite my agreeing with this part of the suggestion), are you saying this based on a technical understanding, a real-world experience understanding, or solely based on a mis-understanding of what is in the manual?

Part of the problem I (and likely others) have with the manual, is that they present these large chunks of configuration files as examples - but don't annotate them to clarify what's being set / why / etc. For example, is there any place that clearly defines:

(for those playing along from home, Appendix B starts on page 171)Again, thank you for the document reference as again, I was looking in the wrong manual (sigh)Interesting cautionary note at the beginning of that section - I'll have to keep that in mind for trouble-shooting issues.I'll have to look through this section more - got to head into the office shortly so I don't have time right now... however a quick perusal shows only cursory information about what needs to be configured and why - and appears to be based solely on the WFM end and not on the various other configuration files (though I might have missed that in the 2 seconds I just took the peruse it)

Adam Stoller

So - the KB article tells you how to fix ODHOME/conf/LiveSite/OpenDeployWebRoot.xml (and odnodes.xml) to use the $targetNode variable as a replicationFarm name instead of an individual node name - but it doesn't tell you what to do about:

I ask most of these questions because I pretty much did the above - (no semi after LSCS_RUNTIME_SEPARATOR) and I can see the information being passed into the deployment in the log file, but it still only seems to deploy to the first node and never to the second one.

HKN1104

[QUOTE]Questions:

No.Thanks

Adam Stoller

[do I just need lscsRuntimeHostname and lscsCanonicalHostname]
For 7.2, That is correct. After LCS_RUNTIME_SEPERATOR, You should be providing lscsruntimeCanonicalHostname and lscsruntimeHostname. Where lscsruntimeHostname comes from teamsite-areadiff.xml and rest of xml that it reads commit, deploy and project.

Okay - I'm at least getting that aspect of it right

[semi-colon after LSCS_RUNTIME_SEPARATOR]
NO, Not necessary

Okay, strange, but okay. Would it hurt if I put one there? (I may find out myself, but I figured I'd ask the question anyway)

[semi-colon before end-double-quote]
Yes, That makes it as correct syntax

Okay - that's missing from the examples - I believe I put it in, but I'll check again in the morning

[is semi-colon after end-double-quote bad?]
Yes, then there is a syntactical mistake, It doesnt know and throws an error saying unable to find that Hostname or Please refer to lscs.log

I'll have to check that too - I don't think I put one there. However, last I checked, the lscs.log wasn't getting written into - but I'll check that again too.

[is end-double-quote on its own line bad?]
No.

Good to know.
For what it's worth - I don't believe I got any deployment errors, but at the same time, it didn't seem to pick up the additional node during the deployment...

Adam Stoller

Okay - so I'm moving forward

Adam Stoller

Well - it looks like the idea of just declaring the lscsRuntimeHostname and lscsRuntimeCanonicalHostname after the LSCS_RUNTIME_SEPARATOR is not sufficient - you need to copy everything else even if it's unchanged (you might be able to trim it down a bit, but I'm not sure its worth the effort to do so)

So I now get things to kick-off the fan-out, but the fan-out itself fails because I declared the target paths in the nodes in the replicationFarm and the path I used is appropriate for some deployments but not for the LSCS deployments.

Sill playing with it ....

Adam Stoller

So I changed the target nodes in the farm, which should work for the ODHOME/conf/lscs/* deployments now - but it turns out that the adaptor is failing to connect to one of the two nodes (the one that wasn't being used before attempting fan-out)

From the ODHOME/log/lscs/adp.lscs-import.teamsite-areadiff<instance>.MYDEFINITIONNAME.<source_host>.to.<target_node>.log:

...
ERROR Exception in import delivery adapter execution
java.net.ConnectionException: Connection refused
    at ...
    ...

I'm guessing it's trying to connect on port 15060 (which is referenced in the same log file as the "Runtime port" - and I'm assuming that's the OOTB default - yes?)

Looking a bit further - this is probably the LSCS and/or LSDS pieces that are supposed to be [at least here] installed / configured through WebSphere - and I don't think that's been done yet on this "other" server... time to get in touch with the IT folks ...

Apparently it was installed / configured - but is currently "stopped" - hopefully when they "start" it, it won't break anything that's currently working and will then let me progress with the fan-out testing to see if I can get that to work...

nipper

If it is failing on the import, then yes that is the LSCS port (typically 1876). Check the lscs.log or the idol.log.

FYI, this error is like the HOPI error. Many things can cause it. Licensing included, mixing case on EAs is another (Title and title).

Adam Stoller

If it is failing on the import, then yes that is the LSCS port (typically 1876). Check the lscs.log or the idol.log.

FYI, this error is like the HOPI error. Many things can cause it. Licensing included, mixing case on EAs is another (Title and title).

I think it's dying during the teamsite-areadiff deployment part (which perhaps is what you refer to as "import"?)

Anyway, we believe we know why it was the way it was (part of a "cluster" but turned off because - at the time - nobody knew how to do fan-out deployments) - hopefully we'll have it "up" tomorrow (though not actively serving the content to the web server) so I can try to find the next obstacle ...

nipper

I think it's dying during the teamsite-areadiff deployment part (which perhaps is what you refer to as "import"?)

Anyway, we believe we know why it was the way it was (part of a "cluster" but turned off because - at the time - nobody knew how to do fan-out deployments) - hopefully we'll have it "up" tomorrow (though not actively serving the content to the web server) so I can try to find the next obstacle ...

OK, then that is different.

It is best to get the receiver logs, and see where the error occurs. Import failures happen in the adp logs.

Adam Stoller

OK, then that is different.

It is best to get the receiver logs, and see where the error occurs. Import failures happen in the adp logs.

It was in the adp log that I saw the failure to connect to the server and the port number that I believe it was using.

all of the lscs/*.xml configs (except one) use the adapter ... Hopefully more news tomorrow.

Adam Stoller

FYI - after combing through some documentation that I had managed to put together based on screen shots of the installations which took place before my involvement with the project, I was able to determine that the process that's [not] listening on port 15060 here is LSDS / "LiveSite Run Time"

Unfortunately, I'll probably have to wait until next week before this gets resolved, in order to proceed with my fun journey into getting TS/OD/LS working in a fan-out model. More time for me to focus on trying to document everything I've done to date...

nipper

FYI - after combing through some documentation that I had managed to put together based on screen shots of the installations which took place before my involvement with the project, I was able to determine that the process that's [not] listening on port 15060 here is LSDS / "LiveSite Run Time"

Unfortunately, I'll probably have to wait until next week before this gets resolved, in order to proceed with my fun journey into getting TS/OD/LS working in a fan-out model. More time for me to focus on trying to document everything I've done to date...

FYI, the process on 15060 should be LSCS runtime not LSDS

HKN1104

It was in the adp log that I saw the failure to connect to the server and the port number that I believe it was using.

all of the lscs/*.xml configs (except one) use the adapter ... Hopefully more news tomorrow.

Ghoti,
In teamsite-areadiff.xml, After /odadpaterSet What do you see. Anything with ReplicationFarmLink, If you see anything inside it, It is good or else add Internal name.

Rest all commit-deploy, rollback and reset-project.xml, files uses different Adapters, Please check whether they are using same adapter or different.

Thanks

HKN1104

Ghoti,
In teamsite-areadiff.xml, After /odadpaterSet What do you see. Anything with ReplicationFarmLink, If you see anything inside it, It is good or else add Internal name.

Rest all commit-deploy, rollback and reset-project.xml, files uses different Adapters, Please check whether they are using same adapter or different.

Thanks

Adding more to this, It should be using ResetProjectDeliveryAdapter for Project.xml. Please check, whether it is using same.

Thanks

Adam Stoller

FYI, the process on 15060 should be LSCS runtime not LSDS

It might be LSCS Runtime - I'm still green in this area, and since I didn't do the installations I'm not entirely sure - is "LiveSite Run Time" LSCS or LSDS?

Adam Stoller

Ghoti,
In teamsite-areadiff.xml, After /odadpaterSet What do you see. Anything with ReplicationFarmLink, If you see anything inside it, It is good or else add Internal name.

Rest all commit-deploy, rollback and reset-project.xml, files uses different Adapters, Please check whether they are using same adapter or different.

Thanks

I'll have to try to check this tomorrow when I'm back in the office.

With regard to team site-areadiff - if after the adapter there's a replicationFarmLink, don't I still have to specify the multiple target nodes in the odAdapter portion? Or are you saying that the odAdapter runs for each node in the replicationFarm?

I don't know which deployment is running which adapter - that's part of what I'll need to check tomorrow - I didn't look that closely and just assumed that since they looked alike (at a glance) that I needed to treat them the same way.

Adam Stoller

Adding more to this, It should be using ResetProjectDeliveryAdapter for Project.xml. Please check, whether it is using same.

Thanks

And if it is - what does that mean relative to what I've posted thus far?
(It probably is, I doubt they modified that deployment configuration file locally)

HKN1104

And if it is - what does that mean relative to what I've posted thus far?
(It probably is, I doubt they modified that deployment configuration file locally)

Typically what autonomy has written for all 4 files (Teamsite-areadiff, rollback, commit-deploy and reset-project xml) uses different class for each node each time it executes. Each node successfully deployed will write to their log files.

When there is another class then, it will overwrite updated log files.

(It probably is, I doubt they modified that deployment configuration file locally)

Yes

Thanks

Adam Stoller

Typically what autonomy has written for all 4 files (Teamsite-areadiff, rollback, commit-deploy and reset-project xml) uses different class for each node each time it executes. Each node successfully deployed will write to their log files.

Is this in response to what you quoted, or the other post where I asked if the odAdapter gets run for each node in the replicant?

HKN1104

I'll have to try to check this tomorrow when I'm back in the office.

With regard to team site-areadiff - if after the adapter there's a replicationFarmLink, don't I still have to specify the multiple target nodes in the odAdapter portion? Or are you saying that the odAdapter runs for each node in the replicationFarm?

Yes, Still we have to give in Adapter. If you are deploying to multiple servers.
If deploying it only to one server. No need to specify in adapter.
I usually do it in this way.
[php]

Below providing Server details

[/php]
After ODadapter. I Specified internal link and give it as "MYFARMNAME-$destinationNode" which will pick respective servers and deploy.
When ever there is change in Destination node or deploying to new server, this will pick it up.

I didn't look that closely and just assumed that since they looked alike (at a glance) that I needed to treat them the same way.

Exactly.

Thanks

HKN1104

Is this in response to what you quoted, or the other post where I asked if the odAdapter gets run for each node in the replicant?

It's for 3 posts earlier.

For that, Adapter runs only once

Thanks

Adam Stoller

It's for 3 posts earlier.

For that, Adapter runs only once

Thanks

I appreciate your responses, but the piecemeal way in which you present them tends to leave me with more questions rather than a feeling of resolution.
To try to answer some of the questions / comments raised:

is where the issue is. If there are any remaining issues on the source side of things I want to try to get them taken care of...