How to automatically deploy new docker images from dockerhub to kubernetes?
Solution 1:
Image reference and manifests
Kubernetes manage deployments, e.g. using a rolling deployment strategy whenever your kind: Deployment manifest is changed.
Now after pushing a commit with dev-* tag, dockerhub creates new image tagged as dev-latest
The main problem with this is that you use the same tag name for every image. This means that your kind: Deployment manifest is not updated when your image is updated, so Kubernetes does not initiate a new rolling deployment, since there are no changes in the manifest.
A good practice is to use an unique image tag every time you push an image, and then update the kind: Deployment manifest with this new image tag name, so Kubernetes automatically trigger a new rolling deployment.
Some image registries can be configured to use "immutable tag names", this means that you are enforced to always use a unique tag name for every built image. Alternatively, you can use the full Image Digest in the kind: Deployment manifest, so that you can have an unique image reference even if the tag name is the same (Image Digest is a hash of the content).
In summary:
- Use unique image tag names for every build
- Update the
kind: Deploymentmanifest with the new tag name - Keep the
kind: Deploymentmanifest (and other yaml manifest) in version control, e.g. Git - Consider use full Image Digest in the manifest when referring to the Image
Process and automation
Updating the manifest and applying it to the cluster can be done in several different ways, but it is recommended to always store the updated manifest in a Git repository after every change, before it is applied to the cluster. In this way, you get good traceability about what has changed and it is easier to get back to a working version if something fail.
With the steps needed as described above, another important practice is to use automation for this. After every change to the Git repository, trigger an automated process to do the work, preferably without any manual steps. Historically, Jenkins has been a popular tool for this, but it has its age and is not designed to run well in a container environment. I now recommend to use a tool like GitHub Actions, Google Cloud Build or a modern system within the Kubernetes cluster, like Tekton Pipelines
Build and Deploy Pipeline using Tekton
If you choose to use Tekton in your Kubernetes cluster, you may structure the project like this:
- Git push of code change to a code repository
- A Tekton Trigger receives an event from the Git system and initiate a new
PipelineRun. - The Tekton Pipeline contains steps for git-clone, build the code, run tests and to build and push an image. The result is an image digest or label. Then you have a last task in the pipeline that, if all previous tasks succeeded, updates the
kind: Deploymentmanifest with the new image-digest and git-push to a repo with the manifests. - A Deployment Pipeline is trigged that apply the manifests to the cluster (perhaps using a deployment strategy of choice, rolling deployment or gradual deployment or canary deployment)
Book recommendations
-
Kubernetes Up & Running 2nd edition (from 2019) - it contains a new chapter: 18. Organizing Your Application that describes how to manage deployments with manifests and version control.
-
Continuous Delivery - a classic book about how to use build and deploy pipelines and automation.
Alternatives
I've discovered some project of CD Foundation: Jenkins X, Spinnaker and Tekton and I guess some of them can help me to implement my idea but I would be great to implement it without one more pet in my zoo.
Deployments on Kubernetes can be done without using declarative manifests and instead be done with imperative commands without any version controlled changes but this is really discouraged for a professional environment. It is important to do this in a declarative and reproducible way although automated pipelines takes some time to setup and configure.