How to make Jenkins agents port available with an Ingress controller on Kubernetes?
I'm trying to setup Jenkins in a Digital Ocean Kubernetes cluster. I'm using a NGINX ingress controller as I want to access my server from a subdomain (jenkins.example.com). Everything is working fine so far, I have my UI on said domain secured with a custom certificate. I started encountering problems when trying to connect my agents (or slaves I read elsewhere?) to the server.
What I've tried
I tried to setup a load balancer but couldn't make it work as I am using Digital Ocean and most of the docs I found were for GKE / EKS and others, which have their own internal LB. I also tried mapping a route /agents
on the service's port name then on another service with a "ClusterIP" type, without success (not found from Jenkins).
My current config
jenkins-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: jenkins-dep
labels:
app: jenkins
spec:
replicas: 1
selector:
matchLabels:
app: jenkins
template:
metadata:
labels:
app: jenkins
spec:
containers:
- name: jenkins
image: jenkins/jenkins:lts-jdk11
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
- containerPort: 50000
jenkins-service.yaml
apiVersion: v1
kind: Service
metadata:
name: jenkins-svc
labels:
app: jenkins
spec:
type: NodePort
selector:
app: jenkins
ports:
- name: ui
protocol: TCP
port: 8080
targetPort: 8080
nodePort: 32500
- name: agents
protocol: TCP
port: 50000
targetPort: 50000
nodePort: 32501
jenkins-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: jenkins-ingress
spec:
tls:
- hosts:
- jenkins.example.com
secretName: tls-secret
rules:
- host: jenkins.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: jenkins-svc
port:
name: ui
#- path: /agents
# pathType: Prefix
# backend:
# service:
# name: jenkins-svc
# port:
# name: agents
The bottom line is: how do I make available the Jenkins port 50000 for the agents to connect on my server? Or how could I change Jenkins settings to accommodate such a config?
Solution 1:
To the best of my knowledge, Jenkins uses a custom protocol for communicating with its build agents that is a binary Java RMI, and not HTTP based. Since the Ingress resource is only for host:
based virtual dispatching of HTTP requests, you cannot use any kind: Ingress
declaration for doing that
But, with the nginx-ingress controller specifically, they do allow exposing TCP services which will likely do what you want