Jenkins: In a declarative jenkins pipeline - can I set the agent label dynamically?
Is there a way to set the agent label dynamically and not as plain string?
The job has 2 stages:
- First stage - Runs on a "master" agent, always. At the end of this stage I will know on which agent should the 2nd stage run.
- Second stage - should run on the agent decided in the first stage.
My (not working) attempt looks like this:
pipeline {
agent { label 'master' }
stages {
stage('Stage1') {
steps {
script {
env.node_name = "my_node_label"
}
echo "node_name: ${env.node_name}"
}
}
stage('Stage2') {
agent { label "${env.node_name}" }
steps {
echo "node_name: ${env.node_name}"
}
}
}
}
The first echo works fine and "my_node_label" is printed. The second stage fails to run on an agent labeled "my_node_label" and the console prints:
There are no nodes with the label ‘null’
Maybe it can help - if I just put "${env}" in the label field I can see that this is a java class as it prints:
There are no nodes with the label ‘org.jenkinsci.plugins.workflow.cps.EnvActionImpl@79c0ce06’
Solution 1:
Here is how I made it: mix scripted and declarative pipeline. First I've used scripted syntax to find, for example, the branch I'm on. Then define AGENT_LABEL variable. This var can be used anywhere along the declarative pipeline
def AGENT_LABEL = null
node('master') {
stage('Checkout and set agent'){
checkout scm
### Or just use any other approach to figure out agent label: read file, etc
if (env.BRANCH_NAME == 'master') {
AGENT_LABEL = "prod"
} else {
AGENT_LABEL = "dev"
}
}
}
pipeline {
agent {
label "${AGENT_LABEL}"
}
stages {
stage('Normal build') {
steps {
echo "Running in ${AGENT_LABEL}"
sh "hostname"
}
}
stage ("Docker build") {
agent{
dockerfile {
dir 'Dockerfiles'
label "${AGENT_LABEL}"
}
}
steps{
sh "hostname"
}
}
}
}
Solution 2:
To see how this works, use a GString object to do a println and return the variable for the agentName at the same time. You can see from the output that this line evaluates well before any of the other pipeline code.
agentName = "Windows"
agentLabel = "${println 'Right Now the Agent Name is ' + agentName; return agentName}"
pipeline {
agent none
stages {
stage('Prep') {
steps {
script {
agentName = "Linux"
}
}
}
stage('Checking') {
steps {
script {
println agentLabel
println agentName
}
}
}
stage('Final') {
agent { label agentLabel }
steps {
script {
println agentLabel
println agentName
}
}
}
}
}
Console output (note that I don't actually have node on this instance labeled Windows, so I aborted after it couldn't find it):
Started by user Admin
[Pipeline] echo
Right Now the Agent Name is Windows
[Pipeline] stage
[Pipeline] { (Prep)
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Checking)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Windows
[Pipeline] echo
Linux
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Final)
[Pipeline] node
Still waiting to schedule task
There are no nodes with the label ‘Windows’
Aborted by Admin
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
ERROR: Queue task was cancelled
Finished: ABORTED
Notice how the line Right Now the Agent Name is Windows appears very early in the output. This explains why your value is null. That statement is evaluated long before your script modifies the variable.
I might try to use a lazy GString to get the variable later.
agentLabel = "${-> println 'Right Now the Agent Name is ' + agentName; return agentName}"
Unfortunately, this throws an error because it is expecting a type of String. Apparently it can coerce the non-lazy GString to a String on its own, but not the lazy version. So when I force coercion to a String, of course, it evaluates the variable at that time (which is again, before the pipeline code actually runs).
agent { label agentLabel as String }
You can solve the problem by falling back to the old node allocation method:
agentName = "Windows"
agentLabel = "${-> println 'Right Now the Agent Name is ' + agentName; return agentName}"
pipeline {
agent none
stages {
stage('Prep') {
steps {
script {
agentName = "Linux"
}
}
}
stage('Checking') {
steps {
script {
println agentLabel
println agentName
}
}
}
stage('Final') {
steps {
node( agentLabel as String ) { // Evaluate the node label later
echo "TEST"
}
script {
println agentLabel
println agentName
}
}
}
}
}
You can see from this console output that it now properly finds the Linux node and finishes the pipeline. The early evaluation while agentName == Windows never happens:
Started by user Admin
[Pipeline] stage
[Pipeline] { (Prep)
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Checking)
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Right Now the Agent Name is Linux
[Pipeline] echo
Linux
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Final)
[Pipeline] echo
Right Now the Agent Name is Linux
[Pipeline] node
Running on Slave 1 in /home/jenkinsslave/jenkins/workspace/test
[Pipeline] {
[Pipeline] echo
TEST
[Pipeline] }
[Pipeline] // node
[Pipeline] script
[Pipeline] {
[Pipeline] echo
Right Now the Agent Name is Linux
[Pipeline] echo
Linux
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] End of Pipeline
Finished: SUCCESS
This would probably work without the lazy GString and type coercion later, but I didn't try that.
Solution 3:
it might be something about the context of the script block.
this works, using a label of 'docker' in second stage:
def hotLabel = 'docker'
pipeline {
agent { label 'master' }
stages {
stage('Stage1') {
steps {
echo "node_name: ${hotLabel}"
}
}
stage('Stage2') {
agent { label "${hotLabel}" }
steps {
echo "node_name: ${hotLabel}"
}
}
}
}
this does not (gets the same There are no nodes with the label ‘null’ error):
def hotLabel = null
pipeline {
agent { label 'master' }
stages {
stage('Stage1') {
steps {
script {
hotLabel = "docker"
}
}
}
stage('Stage2') {
agent { label "${hotLabel}" }
steps {
echo "node_name: ${hotLabel}"
}
}
}
}
Solution 4:
I used a ternary operator to have mine dynamically change.
For below, if the Jenkins pipeline name ends in "prod" the label used is "myagent-prd". Otherwise, it's just "myagent".
def ENVIRONMENT_NAME="${JOB_NAME}".tokenize('-').last().toLowerCase()
pipeline {
agent {
label "myagent${ENVIRONMENT_NAME == "prod" ? "-prd" : "" }"
}