DnR scripts not working

I have a strange situation in that all my dnr scripts suddenly stop working and the only way the situation is resolved is by restarting OpenDeploy. I am on Solaris 8, TeamSite 5.5.2 and OD 6.0.2.1(Build 52162). The logs do not tell me anything significant - i.e., the logs for the same deployment prior and after restart are identical, (the only difference being that the source dnr scripts are now being executed -but the confirmation of that execution does not show up in the logs anyway). This is a production box with over 300 deployments per day and hence needs to be fixed quickly. A support ticket has been opened a month ago.

Any and all suggestions appreciated.

Find more posts tagged with

Comments

Adam Stoller

Are you sure the DNR scripts are failing to run? or are they running and failing (without sending the requisite XML snippet back to OD)?

I think you need to verify which situation is occuring before any more diagnosis can be performed.

If it's the first case - I'm not sure what you can do about it except try to escalate the case through support and work with them.
If it's the second case - you'll need to add a lot more debugging to your script to try to determine the exact cause of failure.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

JackIntheBox

"Are you sure the DNR scripts are failing to run? or are they running and failing (without sending the requisite XML snippet back to OD)?"

Thanks for your reply. I am sure the dnr scripts are not working since I tested this theory by attempting to run simple commands(m, rm of some /tmp files) through dnr's - none work. After a restart though, everything works, including the test dnr scripts. This situation is restricted to source dnr scripts, scripts that run on the target scripts work fine.

Adam Stoller

Perhaps it's a resource issue?
You might need to get your local system administrator involved to see if they can figure out if there is a lack of some resource when the scripts fail to run.
Don't know what else to suggest - sorry.

--fish
Senior Consultant, Quotient Inc.
http://www.quotient-inc.com

JackIntheBox

Fish,

Thanks for your help. Based on your suggestion and of our SA's, I wrote a script to gather the output of /opt/iw-home/bin/iwstat -c and /usr/ucb/ps auxwww (every minute) and am waiting for OpenDeploy to act up. A support ticket has been opened. I will post here about any progress.

Thanks much.

iw_store

Hi ALL,

Even iam facing same problem, when OD completes DNR script is not picking up as result of which my OD is hanging in commit phase as both are transactional.

The solution for this we have is to restart the base server and do the deployments again.

Its not consistant works fine for few days may be 3-4 days and hangs on 5th day.

Logs are not giving any significant message as it DNR.

If any information you get please share, it will help a lot to me.

Sanjeev.

iw_store

Hi ALL,

Even iam facing same problem, when OD completes DNR script is not picking up as result of which my OD is hanging in commit phase as both are transactional.

The solution for this we have is to restart the base server and do the deployments again.

Its not consistant works fine for few days may be 3-4 days and hangs on 5th day.

Logs are not giving any significant message as it DNR.

If any information you get please share, it will help a lot to me.

Sanjeev.

We resolved this issue, the problem is there was a firewall between source and destination. If the deployment goes beyond certain time firewall uses to time out. Due that deployments were irregular. Just to share experience posted this new reply Smiley Happy