How do I make a Jenkins job start after multiple simultaneous upstream jobs succeed?

Solution 1:

Pipeline plugin

You can use the Pipeline Plugin (formerly workflow-plugin).

It comes with many examples, and you can follow this tutorial.

e.g.

// build
stage 'build'
...

// deploy
stage 'deploy'
...

// run tests in parallel
stage 'test'
parallel 'functional': {
  ...
}, 'performance': {
  ...
}

// promote artifacts
stage 'promote'
...

Build flow plugin

You can also use the Build Flow Plugin. It is simply awesome - but it is deprecated (development frozen).

Setting up the jobs

Create jobs for:

• build
• deploy
• performance tests
• functional tests
• promotion

Setting up the upstream

1. in the upstream (here build) create a unique artifact, e.g.:

   echo ${BUILD_TAG} > build.tag
   

2. archive the build.tag artifact.

3. record fingerprints to track file usage; if any job copies the same build.tag file and records fingerprints, you will be able to track the parent.
4. Configure to get promoted when promotion job is successful.

Setting up the downstream jobs

1. I use 2 parameters PARENT_JOB_NAME and PARENT_BUILD_NUMBER

2. Copy the artifacts from upstream build job using the Copy Artifact Plugin

   • Project name = ${PARENT_JOB_NAME}
   • Which build = ${PARENT_BUILD_NUMBER}
   • Artifacts to copy = build.tag

3. Record fingerprints; that's crucial.

Setting up the downstream promotion job

Do the same as the above, to establish upstream-downstream relationship. It does not need any build step. You can perform additional post-build actions like "hey QA, it's your turn".

Create a build flow job

// start with the build
parent = build("build")
parent_job_name = parent.environment["JOB_NAME"]
parent_build_number = parent.environment["BUILD_NUMBER"]

// then deploy
build("deploy")

// then your qualifying tests
parallel (
    { build("functional tests",
          PARENT_BUILD_NUMBER: parent_build_number,
          PARENT_JOB_NAME: parent_job_name) },
    { build("performance tests",
          PARENT_BUILD_NUMBER: parent_build_number,
          PARENT_JOB_NAME: parent_job_name) }
)

// if nothing failed till now...
build("promotion",
    PARENT_BUILD_NUMBER: parent_build_number,
    PARENT_JOB_NAME: parent_job_name)

// knock yourself out...
build("more expensive QA tests",
    PARENT_BUILD_NUMBER: parent_build_number,
    PARENT_JOB_NAME: parent_job_name)

good luck.

Solution 2:

There are two solutions that I have used for this scenario in the past:

1. Use the Join Plugin on your "deploy" job and specify "promote" as the targeted job. You would have to specify "Functional Tests" and "Performance Tests" as the joined jobs and start them via in some fashion, post build. The Parameterized Trigger Plugin is good for this.

2. Use the Promoted Builds Plugin on your "deploy" job, specify a promotion that works when downstream jobs are completed and specify Functional and Performance test jobs. As part of the promotion action, trigger the "promote" job. You still have to start the two test jobs from "deploy"

There is a CRITICAL aspect to both of these solutions: fingerprints must be correctly used. Here is what I found:

1. The "build" job must ORIGINATE a new fingerprinted file. In other words, it has to fingerprint some file that Jenkins thinks was originated by the initial job. Double check the "See Fingerprints" link of the job to verify this.
2. All downstream linked jobs (in this case, "deploy", "Functional Tests" and "Performance tests") need to obtain and fingerprint this same file. The Copy Artifacts plugin is great for this sort of thing.
3. Keep in mind that some plugins allow you change the order of fingerprinting and downstream job starting; in this case, the fingerprinting MUST occur before a downstream job fingerprints the same file to ensure the ORIGIN of the fingerprint is properly set.