There are dozen solutions for performing discovery service in Mesos environment.
We can divide them into 3 groups by the way how client find services:
- Proxy based
- When between clients and service sits proxy e.g., HAProxy (marathon-lb is based on it), fabio, traefik, nixy) that takes care of loadbalancing your services basing on HTTP path, header, domain e.t.c. This solution is the easy to develop and gives opportunity to tune loadbalancing based on request. On the other hand we add additional hop and as a proxy we have MitM situation.
- DNSlike (ask special well known endpoint for location of service)
- Software Defined Network – we can use IP per container with SDN so each container is exposed with unique IP and presents it’s services using default ports 80 for HTTP, 443 for HTTPS and so on. This is most advanced and relatively new techniqe although it uses plain old DNS to find service IP. It could be harder to intorduce then proxy but will work with any type of traffic.
- Service record – where every container is registered in global DNS and client obtain it’s IP and PORT using DNS SRV queries. Consul Mesos DNS provides this type of DNS server. Also some other protocols are based on this idea (take a look at Bonjure). It tries get best of both SDN and proxy. It’s relatively easy to setup and it’s protocol agnostic.
- Other
- Anything that doesn’t fit into other types e.g. inhouse developed solution, etcd or Eureka. It could be deeply tight with infrastructure and application providing some optimizations. It’s worth mention that there are some tries to use DHT based discovery service – Meta Service Discovery
You can find more details about tools that could be used for creating Discovery Service here
We can divide Discovery Services by the way they are populated with service entries:
- Pooling
- Mesos/Marathon is periodically queried about state. This is how Mesos DNS is working. This is easiest method but will cause huge delay beteween service start/stop and changes gets into service discovery. This could be minimized by using healthchecking.
- Event based
- Marathon has ability to push events with information about state changes (There is initative to include event bus int Mesos too — design doc. This way marathon-lb is working. Similar job is done by marathon-consul but data are passed to consul.
- In app/container
- Above sollutions are asynchronus so there could be a timespan when your service discovery state is stale e.g. service started but is not ready to serve requests, or service just died. Even with healtcheck we could not assume all things happen with 0 downtime. Solution to minimize downtime is to let application register itself when it’s ready to serve requests, and deregister before it stop (aka graceful shutdown).