Program to manage process workflow across machines?

Solution 1:

This can be done with RunDeck, using a Job for each separate host step, and a higher-level Job to coordinate those jobs. Although rundeck doesn't have exactly that level of coordination within a workflow, it does allow you to chain together multiple Jobs to get the same effect

E.g:

1. Create a Job for each the actions "run processX on hostY" (JobA-C). These jobs would use the Node Dispatch feature with the correct filter to apply to the hostY only.

2. create a top-level Job with a workflow that uses Job References to execute jobs A-C in order, and not using Node Dispatching.

Running the top-level job would step through and run the other jobs in order, waiting until each one finishes before going to the next step.

You could configure the top-level job to also "keepgoing": that is, even if one step fails, continue to execute all subsequent steps.

You could also configure your lower-level jobs to target multiple nodes in parallel, and the top-level job would still do only one step at a time.

Solution 2:

Skybot Scheduler may be able to do what you want - not 100% certain it can handle the cross-host dependency bits, but it's pretty robust and it's the first thing that jumps to mind.