Jenkins CI Master-Slave setup with docker for slave server

So I have setup up two Ubuntu AWS Instances (a small t2.medium and a big r3.large). The t2.medium has Jenkins CI installed on it and the r3.large has docker engine installed on it. The intention here is to have a Master (t2.medium) server and a Slave (r3.large) server; where the master server can delegate docker containers for jobs (spin up jenkins docker containers as needed).

My problem is that I'm kinda way over my head with this one. I've been researching around but almost anything I do fails on me.

My current situation is: Like I've mentioned above, I have jenkins successfully running on one instance, and docker engine successfully running on the other. I also have the docker plugin, found here, installed on jenkins and I have followed the instructions that are on that page to the teeth. However, when I add a cloud and add credentials the only thing that's showing is "-none-"; however that same credentials shows up at the bottom.

Here's a screenshot of what I'm talking about: enter image description here

The problem with this is that every time I test the connection I get the following error:

shaded.org.apache.http.conn.HttpHostConnectException: Connect to 10.96.24.240:4243 [/10.96.24.240] failed: Connection refused.

Any help would be much appreciated! I've been going at this for more than a week! And I'm just not able to wrap my head around this.

EDIT: If I'm going about it the wrong way, please do guide me on how to achieve the intended setup.

EDIT2: All ports are opened between those two instances.

Here's a screenshot of the security group: enter image description here

EDIT3: Here are my docker images:

ubuntu@ip-10-96-24-240:~$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
ubuntu@ip-10-96-24-240:~$ docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
jenkins-1              latest              9de6ee8eeebc        3 days ago          816.3 MB
ubuntu                 latest              8251da35e7a7        2 weeks ago         188.4 MB
jenkins                latest              0c79d0a394dc        3 weeks ago         888 MB
evarga/jenkins-slave   latest              8880612971b0        8 months ago        610.8 MB

Ok so this doesn't fully satisfy my problem, however it is a viable answer to the shaded.org.apache.http.conn.HttpHostConnectException: Connect to 10.96.24.240:4243 [/10.96.24.240] failed: Connection refused problem. My problem was that I didn't add the following code to /etc/init/docker.conf:

description     "Docker daemon"

start on filesystem and started lxc-net
stop on runlevel [!2345]

respawn

script
    /usr/bin/docker -H tcp://127.0.0.1:4243 -d
end script

I also put 1.8.1 for the Docker Version which is the client version. Instead it should be the API version which is currently 1.20.

I hope this helps any beginner like me. :)

EDIT: Docker conf file has changed a little bit since I've posted this, here's how my docker.conf looks like now:

pre-start script
        # see also https://github.com/tianon/cgroupfs-mount/blob/master/cgroupfs-mount
        if grep -v '^#' /etc/fstab | grep -q cgroup \
                || [ ! -e /proc/cgroups ] \
                || [ ! -d /sys/fs/cgroup ]; then
                exit 0
        fi
        if ! mountpoint -q /sys/fs/cgroup; then
                mount -t tmpfs -o uid=0,gid=0,mode=0755 cgroup /sys/fs/cgroup
        fi
        (
                cd /sys/fs/cgroup
                for sys in $(awk '!/^#/ { if ($4 == 1) print $1 }' /proc/cgroups); do
                        mkdir -p $sys
                        if ! mountpoint -q $sys; then
                                if ! mount -n -t cgroup -o $sys cgroup $sys; then
                                        rmdir $sys || true
                                fi
                        fi
                done
        )
end script

script
        # modify these in /etc/default/$UPSTART_JOB (/etc/default/docker)
        DOCKER=/usr/bin/$UPSTART_JOB
        DOCKER_OPTS='-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock'
        if [ -f /etc/default/$UPSTART_JOB ]; then
                . /etc/default/$UPSTART_JOB
        fi
        exec "$DOCKER" daemon $DOCKER_OPTS
end script

# Don't emit "started" event until docker.sock is ready.
# See https://github.com/docker/docker/issues/6647
post-start script
        DOCKER_OPTS='-H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock'
        if [ -f /etc/default/$UPSTART_JOB ]; then
                . /etc/default/$UPSTART_JOB
        fi
        if ! printf "%s" "$DOCKER_OPTS" | grep -qE -e '-H|--host'; then
                while ! [ -e /var/run/docker.sock ]; do
                        initctl status $UPSTART_JOB | grep -qE "(stop|respawn)/" && exit 1
                        echo "Waiting for /var/run/docker.sock"
                        sleep 0.1
                done
                echo "/var/run/docker.sock is up"
        fi
end script

So the only thing that needs to be done now is to just add -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock to DOCKER_OPTS

Jenkins CI Master-Slave setup with docker for slave server

Related

Recent Posts