Docker multi-host networking

If you ever have a need to run dockerized workload at scale of more than one host, but not large enough to bother with stuff like Kubernetes, chances are you'll be dealing with multi-host networking, i.e. containers situated on one host need to be able to talk with one on another host.

Of course the official document is where you want to start. But the part where they introduce Machine seems a bit extravagant, as if they were trying to promote one of their products here. Also if you already have your docker infrastructure installed, you'll be facing the necessity of modifying your setup and adding a key-value store, instead of wiping out what you've done to start all over again with the fancy Machine.

What they did with Machine in the document was simply installing docker - what you probably already have - and configuring it to talk to a key-value store. The configuration part was just adding --cluster-store and --cluster-advertise options when running dockerd.

If you are on Ubuntu, you'll want to modify /etc/default/docker and make sure the following line is enabled.

DOCKER_OPTS="--cluster-store=consul://172.16.10.31:8500 --cluster-advertise=eth0:2376"

It goes without saying that the detailed values need to be adjusted according to your own environment.

Restart dockerd to activate the new configurations. Repeat these on every docker host and you are all set to create an overlay network across your whole infrastructure.

Update: from Ubuntu 16.04 onward you may want to check /etc/docker/daemon.json instead of /etc/default/docker for docker daemon options.