nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Workflow tab current task exlamation point

SethHickel

Can someone tell me what the clickable "!" on the active tasks does? Is it to move the priority of a task up or down if there are more then one task?

Trying to find information on how to retry/restart tasks.

Find more posts tagged with

Comments

Pawan

Can someone tell me what the clickable "!" on the active tasks does? Is it to move the priority of a task up or down if there are more then one task?

Trying to find information on how to retry/restart tasks.

Its not to change the priority but to sort the active task(s) in ascending/descending order of their priority.

SethHickel

Thanks.

So - is there a way to re-start or re-try a task. As an example, one that stalled due to time out or some other reason?

Pawan

Do you mean making an inactive task active, right?
It can be done in your workflow, are you talking about any specific task?

nipper

Usually not, but there can be.

One reason I always write a WFT from scractch rather than hacking a default one is then I can add all the timeout/restart capabilities I want.

SethHickel

OK - so is the solution to end the job and redeploy? It seems that even after ending jobs new jobs using the same workflow stall on the actual deploy task.

Pawan

You have to update the WFT file as nipper said, starting another new job will serve the same behavior. What I understand from you post is you want to redeploy if the deployment fails, for that you need to link the deployment task to itself on failure or after any notification if you want to do so.

SethHickel

Thanks - we will have developers look at working on the WFT. The issue I have now is that even after killing jobs that are stalled and starting a new workflow the new jobs stall. Other workflows and deployments are working fine. This is specific two workflows that deploy to the servers that those jobs that stalled do.

Not sure why new jobs continue to fail - all services up and healthy - servers are up and healthy. Looking at logs to try and find something ...

Pawan

1. Check for OD services running on both receivers.
2. Check the OD logs on both base and receivers.
3. For future, have a timeout for your deployment task, so they do not stall.

You may start another thread as this topic is no where related to the subject of this thread.

nipper

Thanks - we will have developers look at working on the WFT. The issue I have now is that even after killing jobs that are stalled and starting a new workflow the new jobs stall. Other workflows and deployments are working fine. This is specific two workflows that deploy to the servers that those jobs that stalled do.

Not sure why new jobs continue to fail - all services up and healthy - servers are up and healthy. Looking at logs to try and find something ...

You need some debugging in the OD tasks. Does it even try to start the deployment ?

Here is a trick I use

Run the WF, wait for it to hang on the deploy task,

open a shell, run

iwperl ...path/to/deploy.ipl INSERT-JOB# INSERT-TASK# /path/to/workarea

substituting the IPL, JobID, TaskID and workarea path for the job that is hung. You will see it run and can much more easily catch runtime errors.

Adam Stoller

OK - so is the solution to end the job and redeploy? It seems that even after ending jobs new jobs using the same workflow stall on the actual deploy task.

You need to determine if the deploy task script / program is failing to run (and hence the task is hanging because there's no timeout handler) or whether the deployment is being initiated and is, itself, hanging (which again could be caught by a timeout handler).
If this is a custom script / program - you should make sure that it logs diagnostics somewhere.
If this is an OOTB script / program - you need to check all the possible log files - add debugging if you can, or replace the OOTB script/program with your own custom one and then ... (see above).

US_Map.zip

nipper

You need to determine if the deploy task script / program is failing to run (and hence the task is hanging because there's no timeout handler) or whether the deployment is being initiated and is, itself, hanging (which again could be caught by a timeout handler).
If this is a custom script / program - you should make sure that it logs diagnostics somewhere.
If this is an OOTB script / program - you need to check all the possible log files - add debugging if you can, or replace the OOTB script/program with your own custom one and then ... (see above).

Seth's company has done waht way too many people do & that is copy the configurable_default_submit WF to their on and customized it to work in their environment. That is a bad idea (IMHO) a custom one can be written without much trouble and fit the needs much better.

ISCBorisB

OK - so is the solution to end the job and redeploy? It seems that even after ending jobs new jobs using the same workflow stall on the actual deploy task.

Cavalier locking is another reason that often causes "hung" workflows.

What is the "lock" attribute of the stuck task? If attached Files are already locked, WF Engine will try again and
again(after some timeout), possibly indefinitely. If that's the case, set 'lock' to false and use dedicated 'locktask'
instead. This way you can determine locking Success or Failure and fork your Workflow to process these outcomes
accordingly.

SethHickel

Seth's company has done waht way too many people do & that is copy the configurable_default_submit WF to their on and customized it to work in their environment. That is a bad idea (IMHO) a custom one can be written without much trouble and fit the needs much better.

Yes - true and noted. :-)

Cavalier locking

Well - I could see this as an issue if we were seeing this on all/more deployments - it occurs mainly when a large number of files or large files are deployed. I'm thinking it is more of a time out issue.

So - I think we need to increase what we log and try and find some threshold for these timeout issues (maybe find out why they are occuring) and see if we can provide some workflow task that the users can kick off to either (or both) remove some files and restart the task.

nipper

Check the deploy task in the WF you are having the problem and see if it has a timeout. I looked through one of your others and many tasks have timeouts (but not the deploy task).

It certainly *sounds* like it is a timeout issue.

You also may want to reduce OD logging, by default it is verbose and chews up a lot of disk and time. That is done on both the sender and receiver.

Adam Stoller

Yes - true and noted. :-)

Well - I could see this as an issue if we were seeing this on all/more deployments - it occurs mainly when a large number of files or large files are deployed. I'm thinking it is more of a time out issue.

So - I think we need to increase what we log and try and find some threshold for these timeout issues (maybe find out why they are occuring) and see if we can provide some workflow task that the users can kick off to either (or both) remove some files and restart the task.

If it's a large number of files (e.g.: > 300) - it sounds like a case where iw.cfg wasn't "fixed" to not include files on the command line for external task scripts.

If it's fewer ( e.g.: < 100 files) larger files - then it may just be a timeout issue, though I would suggest checking the network bandwidth between sender and receiver and look at the various options being used in the deployment that might cause performance issues.

ISCBorisB

If it's a large number of files (e.g.: > 300) - it sounds like a case where iw.cfg wasn't "fixed" to not include files on the command line for external task scripts.

If it's fewer ( e.g.: < 100 files) larger files - then it may just be a timeout issue, though I would suggest checking the network bandwidth between sender and receiver and look at the various options being used in the deployment that might cause performance issues.

I was hoping that this issue has been covered [post=146142]here[/post] - but who knows...

log.txt

nipper

It was but because the iwhome/tmp/cci folder was deleted WFTs failed and he rolled some changes back. So it is highly likely that the specified change is not there.