After discovering that this is massively undocumented,here’s the correct way to mount a NFS volume using stack and docker compose.
The most important thing is that you need to be using version: "3.2"
or higher. You will have strange and un-obvious errors if you don’t.
The second issue is that volumes are not automatically updated when their definition changes. This can lead you down a rabbit hole of thinking that your changes aren’t correct, when they just haven’t been applied. Make sure you docker rm VOLUMENAME
everywhere it could possibly be, as if the volume exists, it won’t be validated.
The third issue is more of a NFS issue – The NFS folder will not be created on the server if it doesn’t exist. This is just the way NFS works. You need to make sure it exists before you do anything.
(Don’t remove ‘soft’ and ‘nolock’ unless you’re sure you know what you’re doing – this stops docker from freezing if your NFS server goes away)
Here’s a complete example:
[root@docker docker-mirror]# cat nfs-compose.yml
version: "3.2"
services:
rsyslog:
image: jumanjiman/rsyslog
ports:
- "514:514"
- "514:514/udp"
volumes:
- type: volume
source: example
target: /nfs
volume:
nocopy: true
volumes:
example:
driver_opts:
type: "nfs"
o: "addr=10.40.0.199,nolock,soft,rw"
device: ":/docker/example"
[root@docker docker-mirror]# docker stack deploy --with-registry-auth -c nfs-compose.yml rsyslog
Creating network rsyslog_default
Creating service rsyslog_rsyslog
[root@docker docker-mirror]# docker stack ps rsyslog
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
tb1dod43fe4c rsyslog_rsyslog.1 jumanjiman/rsyslog:latest swarm-4 Running Starting less than a second ago
[root@docker docker-mirror]#
Now, on swarm-4:
root@swarm-4:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
d883e0f14d3f jumanjiman/rsyslog:latest "rsyslogd -n -f /e..." 6 seconds ago Up 5 seconds 514/tcp, 514/udp rsyslog_rsyslog.1.tb1dod43fe4cy3j5vzsy7pgv5
root@swarm-4:~# docker exec -it d883e0f14d3f df -h /nfs
Filesystem Size Used Available Use% Mounted on
:/docker/example 7.2T 5.5T 1.7T 77% /nfs
root@swarm-4:~#
This volume will be created (but not destroyed) on any swarm node that the stack is running on.
root@swarm-4:~# docker volume inspect rsyslog_example
[
{
"CreatedAt": "2017-09-29T13:53:59+10:00",
"Driver": "local",
"Labels": {
"com.docker.stack.namespace": "rsyslog"
},
"Mountpoint": "/var/lib/docker/volumes/rsyslog_example/_data",
"Name": "rsyslog_example",
"Options": {
"device": ":/docker/example",
"o": "addr=10.40.0.199,nolock,soft,rw",
"type": "nfs"
},
"Scope": "local"
}
]
root@swarm-4:~#