How Things Work

Service Discovery & Load Balancing

How services find each other when addresses are ephemeral — the registry, self-registration with health checks, server-side vs client-side discovery, and load balancing across live instances.

How It Works

In a dynamic microservice environment, instances come and go constantly, so their network addresses can't be hard-coded. Service discovery solves this with a registry: instances register themselves and heartbeat, and callers look up the current healthy set at runtime, then load-balance across it. Kubernetes builds this in via Services and DNS; standalone systems use Consul, Eureka, or etcd.

The Problem: Addresses Change

In a dynamic system, service instances are constantly created, destroyed, moved, and rescheduled by autoscalers and orchestrators. Their IPs and ports are ephemeral, so hard-coding addresses (or even a static list) breaks immediately. Callers need a way to find current instances at runtime.

The Service Registry

A registry (Consul, Eureka, etcd, or Kubernetes' built-in) is the live phone book. Each instance registers itself on startup with its address and a health check, and sends heartbeats. Instances that stop heart-beating, or fail their health check, are automatically removed — so the registry always reflects who's actually alive.

Server-Side vs Client-Side Discovery

Server-side: callers hit a load balancer (or the platform's Service), which queries the registry and forwards the request — clients stay simple. Client-side: the caller queries the registry itself and picks an instance (e.g. with a library like Ribbon), saving a hop but pushing discovery logic into every client.

Load Balancing Across Instances

Once the healthy set is known, requests are spread across it — round robin, least connections, or weighted. Combined with health checks, this means traffic only ever goes to live instances, and adding capacity is as simple as starting a new instance that registers itself.

Key Concepts

📖Service Registry

A live directory of service instances and their addresses, kept current by registration + heartbeats/health checks.

✍️Self-Registration

Instances register themselves on startup and deregister (or expire) on shutdown/failure.

🖥️Server-Side Discovery

Clients call a load balancer that consults the registry and forwards — clients stay dumb. (Kubernetes Services do this.)

📱Client-Side Discovery

The client queries the registry and chooses an instance itself — one fewer hop, more client logic.

🩺Health Check

Periodic probe (/health) that removes failed instances from the registry so traffic avoids them.

💓Heartbeat / TTL

Instances renew their registration periodically; a missed heartbeat expires the entry automatically.

Registry lookup & registration

tsx

1// ❌ Hard-coded address — breaks the moment the instance moves or scales
2const orders = await http.get("http://10.0.3.7:5001/orders");
3
4// ✅ Discover via the registry, then load-balance across healthy instances
5const instances = await registry.lookup("orders-service");   // [:5001, :5002]
6const target = loadBalancer.pick(instances);                 // round-robin
7const orders = await http.get(`${target}/orders`);
8
9// Each instance registers itself and heartbeats so the registry stays current:
10await registry.register({
11  service: "orders-service",
12  address: `${HOST}:${PORT}`,
13  check: { http: "/health", interval: "5s", deregisterAfter: "30s" },
14});
15
16// In Kubernetes this is built in: a Service gives a stable DNS name + virtual IP
17//   http://orders-service.default.svc.cluster.local
18// and kube-proxy load-balances to the healthy Pods automatically.

💡

Why This Matters

Service discovery is foundational plumbing for any microservice or autoscaled system, and it underlies how Kubernetes networking works. Understanding registries, health checks, and the client-side vs server-side trade-off is essential for designing systems that scale and self-heal — and it's a common interview topic once the conversation reaches inter-service communication.

Common Pitfalls

⚠Hard-coding addresses or static lists in a dynamic environment — they go stale on every scale/reschedule.

⚠Discovery without health checks, so the registry hands out dead instances.

⚠Stale registry caches on the client side — a too-long TTL keeps routing to gone instances.

⚠The registry itself being a single point of failure — it must be clustered/highly available.

⚠Reinventing discovery on Kubernetes, which already provides Services, DNS, and load balancing out of the box.

Real-World Use Cases

1Autoscaling Breaks Hard-Coded Endpoints

Scenario

A service calls 'inventory' at a fixed IP from config. When the inventory service autoscales (new pods on new IPs) or gets rescheduled to another node, calls start failing because the configured address no longer exists.

Problem

Static addresses can't survive a dynamic environment where instances are ephemeral. Every scale event or reschedule invalidates the config, causing outages and forcing manual updates.

Solution

Register every inventory instance in a service registry (or rely on Kubernetes Services for a stable DNS name + virtual IP). Callers look up 'inventory-service' and load-balance across whatever instances are currently healthy — scaling and rescheduling become transparent.

💡

Takeaway: Never hard-code instance addresses in a dynamic system. Discover them through a registry (or platform DNS) so callers always reach the current, healthy set regardless of how instances come and go.

2Traffic to a Dead Instance

Scenario

An instance crashes but callers keep sending requests to it for minutes, returning errors, because nothing removed it from the pool of targets.

Problem

Without health checking tied to discovery, the list of targets includes dead instances. Load balancing blindly across a stale list sends a share of traffic into the void.

Solution

Configure health checks (HTTP /health with an interval and deregister-after window) so the registry detects the failure within seconds and removes the instance. Load balancers then only ever pick from the live set, and the crashed instance receives no further traffic.

💡

Takeaway: Discovery and health checking go together. A registry that actively prunes unhealthy instances ensures load balancing routes only to live targets — turning a crash into a non-event instead of a partial outage.

Join the discussion

Comments

Loading comments...

Cookie preferences

We use essential cookies for sign-in and preferences. With your permission, we also use basic analytics to improve lessons.