Share disks through NFS on a CoreOS cluster?

Is it possible for a CoreOS cluster to share disk space, e.g. using NFS? If so, how would one go about this? (e.g. in the scenario where one node has a lot of disk space). This would be useful to avoid having each node have to download and store it's own library of docker images, for instance, or to share home directory space across nodes.

Because we cannot install additional software directly in CoreOS, I imagine one would have to write a container just to install NFS (e.g. nfs-kernel-server on a Ubuntu-based container).

I have no idea if this is possible, but am hoping there's some established way to go about sharing disk space across a CoreOS cluster (after all, seems like a common expectation for a cluster, and perhaps my proposal below is more convoluted than necessary). Just to provide some fodder for feedback, here's what I'm thinking so far:

Providing the host-side of NFS seems like a reasonable docker task, e.g. I imagine a Dockerfile like:

FROM ubuntu:14.04
ENV CLIENT_IP 11.111.111.111          
RUN apt-get update && apt-get install -y nfs-kernel-server supervisor
RUN mkdir /var/nfs && chown nobody:nogroup /var/nfs
RUN echo "/home       ${CLIENT_IP}(rw,sync,no_root_squash,no_subtree_check)" >> /etc/exports
RUN echo "/var/nfs    ${CLIENT_IP}(rw,sync,no_subtree_check)" >> /etc/exports
RUN exportfs -a
CMD service nfs-kernel-server start

Where CLIENT_IP was filled out appropriately (and perhaps we need to replace the CMD with a call to supervisord or similar to make this persistent, but you get the idea)

So, how would we link volumes appropriately when running this container? Which volume would we link from the CoreOS host? Or do I need to add something like --net="host" to make the client available?

docker run -v /home:/home nfs-server

It's not at all clear to me how we could implement the client side of things, since once again we would need a container to provide nfs-common, and somehow figure out how other containers could then share that resource (perhaps some appropriate use of --volumes-from?) I'd love to see an outline of how to go about this, or why it's not possible & if there are better alternatives to address this use case. Thanks!

You should be able to mount an NFS volume on the CoreOS host. If there is no userland tools to export a filesystem from CoreOS, you can use Fedora toolbox container provided by the toolbox command.

You need to start rpc-mountd and nfsd systemd services and define your nfs exports in /etc/exports file.

Example of cloud-config.yml:

coreos:
  units:
    - name: rpc-mountd.service
      command: start
      enable: true
    - name: nfsd.service
      command: start
      enable: truestrong text
write_files:
  - path: /etc/exports
    permissions: '0644'
    # Change /network-raid with the dir you want to export over nfs
    content: /network-raid/ 192.168.1.0/24(rw,async,no_subtree_check,no_root_squash,fsid=0)

I have this setup working with CoreOS v723.3.0.

You can find more about sharing disk through NFS on CentOS on the following links:

https://groups.google.com/forum/#!topic/coreos-user/-5_r8sAqyPk
http://blog.scottlowe.org/2015/02/20/config-mount-nfs-coreos/

Share disks through NFS on a CoreOS cluster?

Related

Recent Posts