Service Discovery

How microservices find each other at runtime — without hardcoded IP addresses.

What you will learn

• Understand why hardcoded addresses fail in dynamic environments
• Differentiate client-side vs server-side service discovery
• Know how health checks keep the registry accurate
• Recognise real-world tools: Consul, Eureka, Kubernetes DNS

Service Discovery

You have a checkout service that needs to call the inventory service. You deployed three inventory instances across two regions.

What IP address does checkout call?

If you hardcode 10.0.1.42, you get:

• Monday: works.
• Tuesday: instance scaled down, new IP is 10.0.1.67. Checkout times out.
• Wednesday: you update the config. Deploy. Repeat forever.

Service Discovery solves this. Services register their location at startup. Clients look up the location at call time. The registry is always current.

1. The Registry: The Phone Book of Your System

A service registry is a database of running service instances. Each entry holds:

Services register on startup. They maintain their registration by sending heartbeats — periodic keep-alive signals. If heartbeats stop (crash, OOM kill, network partition), the registry marks the instance unhealthy and removes it after the TTL expires.

This means the registry is self-healing. No operator needed to clean up crashed instances.

2. Client-Side Discovery

The client owns the lookup and the load-balancing decision.

1. Client → Registry: GET /api-svc/instances
2. Registry → Client: ["10.0.1.42:8080", "10.0.1.43:8080"]
3. Client picks one (round-robin, random, etc.)
4. Client → 10.0.1.42:8080: POST /api/order
5. 10.0.1.42:8080 → Client: 200 OK

Used by: Netflix Eureka + Ribbon (Java/Spring ecosystem)

Pros:

• Client can apply custom load-balancing logic (latency-aware, zone-aware)
• One fewer network hop — no intermediary proxy
• Simple registry: it's just a data store

Cons:

• Every client language/framework needs a discovery library
• Registry API becomes a coupling point — changing it requires updating all clients
• Client must handle retry and failover logic itself

3. Server-Side Discovery

The client is kept dumb. It only knows the load balancer's address.

1. Client → Load Balancer: POST /api/order
2. LB → Registry: GET /api-svc/instances
3. Registry → LB: ["10.0.1.42:8080", "10.0.1.43:8080"]
4. LB → 10.0.1.42:8080: POST /api/order (forwarded)
5. 10.0.1.42:8080 → LB → Client: 200 OK

Used by: AWS ALB + ECS, Kubernetes (kube-proxy + CoreDNS), HAProxy + Consul Template

Pros:

• Client has zero discovery logic — works with any language
• Centralised routing policy: timeouts, retries, circuit-breaking all in one place
• Services can move without clients noticing

Cons:

• Extra network hop through the load balancer
• LB becomes a critical path component (needs HA setup)
• More infrastructure to maintain

4. Health Checks: Keeping the Registry Honest

A registry with stale entries is worse than no registry — it routes traffic to dead instances.

There are two health check patterns:

Heartbeat / TTL-based (push) The service sends a keep-alive every N seconds. If the registry doesn't hear from it within TTL, it evicts the entry. Consul and Eureka both support this. Simple and low-overhead.

Active health probes (pull) The registry (or LB) actively hits a /health endpoint on each registered instance on a schedule. More accurate — detects partial failures like "process is up but serving 500s" — but adds traffic and complexity.

Production systems often combine both: heartbeats for fast deregistration on crash, active probes for detecting degraded-but-alive instances.

5. Kubernetes DNS: Discovery Without a Registry Client

Kubernetes takes a different approach. Services get a stable DNS name:

inventory-svc.default.svc.cluster.local

Under the hood, kube-proxy watches the API server for endpoint changes and updates iptables rules. CoreDNS resolves the service name to a cluster IP, and kube-proxy distributes connections across healthy pods.

The application just calls http://inventory-svc — no SDK, no registry API. Discovery is infrastructure.

This is server-side discovery at the platform level.

6. Choosing the Right Pattern

7. What Happens When the Registry Goes Down?

A registry that's a single point of failure is a liability. Production registries solve this with:

• Clustering: Consul uses Raft consensus for a replicated registry. Zookeeper uses ZAB.
• Client-side caching: Clients cache the last known instance list. Stale but available.
• Sidecar pattern: The Envoy proxy sidecar in service meshes holds a local copy of the registry state, refreshed asynchronously.

The key insight: discovery should degrade gracefully. A stale registry entry that sometimes routes to a dead instance is better than a discovery failure that takes down all traffic.

Key Takeaways

• Services register their address + health at startup via heartbeats; the registry auto-evicts on TTL expiry
• Client-side: client queries registry, picks instance, connects directly — more control, more coupling
• Server-side: client talks to LB, LB handles lookup — simpler clients, one more hop
• Kubernetes DNS is server-side discovery abstracted to the platform layer
• Registries need HA (Raft/ZAB) and clients need caching to survive registry outages