How to mimic '--volumes-from' in Kubernetes

I'm looking for a pattern that allows to share volumes between two containers running on the same pod in Kubernetes.

My use case is: I have a Ruby on Rails application running inside a docker container. The docker image contains static assets in /app/<app-name>/public directory, and I need to access those assets from the nginx container running alongside in the same pod.

In 'vanilla' docker I would have used --volumes-from flag to share this directory:

docker run --name app -v /app/<app-dir>/public <app-image>
docker run --volumes-from app nginx

After reading this doc: https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/volumes.md I tried this (only relevant entries presented):

spec:
  containers:
    - image: <app-image>
      name: <app-name>
      volumeMounts:
        - mountPath: /app/<app-name>/public
          name: assets
    - image: nginx
      name: nginx
      volumeMounts:
        - mountPath: /var/www/html
          name: assets
          readOnly: true
    volumes:
      - name: assets
        hostPath:
          path: /tmp/assets

But:

• Even though /tmp/assets on the node exists, it's empty
• /app/<app-name>/public inside the app container is also empty

As a workaround I'm gonna try to populate the shared directory when the application container is up (simply cp /app/<app-name>/public/* to shared directory), but I really dislike this idea.

Question: how to mimic --volumes-from in Kubernetes, or if there is no direct counterpart, how can I share files from one container to other running in the same pod ?

apiVersion: v1beta3

Client Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}
Server Version: version.Info{Major:"0", Minor:"17", GitVersion:"v0.17.0", GitCommit:"82f8bdac06ddfacf493a9ed0fedc85f5ea62ebd5", GitTreeState:"clean"}

Solution 1:

[update-2016-8] In latest Kubernetes release, you can use a very nice feature named init-container to replace the postStart part in my answer below, which will make sure the container order.

apiVersion: v1
kind: Pod
metadata:
  name: javaweb-2
spec:
  initContainers:
  - name: war
    image: resouer/sample:v2
    command: ["cp", "/sample.war", "/app"]
    volumeMounts:
    - mountPath: /app
      name: app-volume
  containers:
  - name: tomcat
    image: resouer/mytomcat:7.0
    command: ["sh","-c","/root/apache-tomcat-7.0.42-v2/bin/start.sh"]
    volumeMounts:
    - mountPath: /root/apache-tomcat-7.0.42-v2/webapps
      name: app-volume
    ports:
    - containerPort: 8080
      hostPort: 8001
  volumes:
  - name: app-volume
    emptyDir: {}

NOTE: initContainer is still a beta feature so the work version of this yaml is actually like: http://kubernetes.io/docs/user-guide/production-pods/#handling-initialization, please notice the pod.beta.kubernetes.io/init-containers part.

---original answer begin---

Actually, you can. You need to use container life cycle handler to control what files/dirs you want to share with other containers. Like:

---
apiVersion: v1
kind: Pod
metadata:
    name: server
spec:
    restartPolicy: OnFailure
    containers:
    - image: resouer/sample:v2
      name: war
      lifecycle:
        postStart:
          exec:
            command:
              - "cp"
              - "/sample.war"
              - "/app"
      volumeMounts:
      - mountPath: /app
        name: hostv1 
    - name: peer
      image: busybox
      command: ["tail", "-f", "/dev/null"]
      volumeMounts:
      - name: hostv2
        mountPath: /app/sample.war
    volumes:
    - name: hostv1
      hostPath:
          path: /tmp
    - name: hostv2
      hostPath:
          path: /tmp/sample.war

Please check my gist for more details:

https://gist.github.com/resouer/378bcdaef1d9601ed6aa

And of course you can use emptyDir. Thus, war container can share its /sample.war to peer container without mess peer's /app directory.

If we can tolerate /app been overridden, it will be much simpler:

---
apiVersion: v1
kind: Pod
metadata:
  name: javaweb-2
spec:
  restartPolicy: OnFailure
  containers:
  - image: resouer/sample:v2
    name: war
    lifecycle:
      postStart:
        exec:
          command:
            - "cp"
            - "/sample.war"
            - "/app"
    volumeMounts:
    - mountPath: /app
      name: app-volume
  - image: resouer/mytomcat:7.0
    name: tomcat
    command: ["sh","-c","/root/apache-tomcat-7.0.42-v2/bin/start.sh"]
    volumeMounts:
    - mountPath: /root/apache-tomcat-7.0.42-v2/webapps
      name: app-volume
    ports:
    - containerPort: 8080
      hostPort: 8001 
  volumes:
  - name: app-volume
    emptyDir: {}

Solution 2:

The answer is - for now - you can't. Here's a couple of discussion threads from the Kubernetes issues:

• https://github.com/GoogleCloudPlatform/kubernetes/issues/6120
• https://github.com/GoogleCloudPlatform/kubernetes/issues/831

However, may I suggest that you have an alternate design that might work better?

1. If your assets are locked at the point of the container going live, you could use something like gitRepo volume which would copy it to an emptyDir at the point of going live, and would mean you wouldn't have to move the content around at all, just download it directly to the shared directory.
2. If your assets are locked at the point of the container being built, it's probably best to copy them in at that point, using the Docker COPY command.
3. If you really want to stick with the way you're doing it, you would have to copy the content to the emptyDir volume, which is designed for exactly what you're looking for (minus the lack of having to copy it in).

NFS[1] volumes also could solve your problem, but may be overly complex.

Additionally, I'd recommend that these two services exist in different pods, so you can scale each separately. You can create a service endpoint to communicate between them if you need to.

[1] https://github.com/GoogleCloudPlatform/kubernetes/blob/master/examples/nfs/nfs-web-pod.yaml

Solution 3:

Further update from the future:

There is now a FlexVol plugin for Docker volumes: https://github.com/dims/docker-flexvol

At the time of writing, FlexVol is still an alpha feature though, so caveat emptor.

Solution 4:

Kubernetes has its own volume types and these are most used volume type:

1. emptyDir
2. secret
3. gitRepo
4. hostPath (similar to --volumes-from)
5. config Maps
6. persistent storage (storage disks provided by cloud platforms)

You can find more about kubernets volumes here -https://kubernetes.io/docs/concepts/storage/volumes/

an example of hostpath volume :

apiVersion: v1
kind: Pod
metadata:
  name: test-pd
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /test-pd
      name: test-volume
  volumes:
  - name: test-volume
    hostPath:
      # directory location on host
      path: /data
      # this field is optional
      type: Directory

hostpath will mount host/node directory to container directory.Multiple containers inside a pod can use different or same volumes.You need to mention it in each container. hostPath volumes are independent of pod lifecycle but it create tight coupling between node and pod , you should avoid using hostPath.