Anycast Routing and DNS Steering Diverge Under Load
Anycast and DNS steering both promise to put users near healthy capacity. They do it through different control planes, and under load those planes diverge in ways that matter. This piece breaks down what each genuinely gets right, where each fails, and the hybrid deployment pattern that holds up when traffic spikes and health checks start to flap.
Two steering planes, one failure mode
Anycast assigns the same IP address to devices in multiple locations. Routers forward packets toward the nearest instance using normal BGP decision-making, typically lowest hop count. DNS steering takes the opposite approach: a resolver returns different answers based on policy, geography, or weighted records. Both are proximity mechanisms. Neither is a load balancer in the traditional sense.
The divergence starts when the two planes disagree about health. BGP withdrawal is binary and fast. Application health is graded and often slow to detect. A point of presence can be saturated while its BGP session still advertises the prefix, and a DNS health check can lag real capacity by minutes. That gap is where users land on degraded infrastructure.
Treat the two as separate control planes with separate failure budgets. Anycast fails closed on path loss and open on capacity. DNS steering fails open on cache expiry and closed on policy misconfiguration. Knowing which direction each fails is more useful than comparing their nominal latency numbers.
Anycast's real win is statelessness
BGP convergence handles path failure without application logic. When a route is withdrawn, routers reconverge in seconds and traffic shifts to the next nearest instance. No session affinity problem exists because anycast is best suited to stateless services: DNS, TLS termination, cache fills, and connectionless request routing. There is no per-user state to migrate.
That statelessness is the design constraint, not a side effect. Anycast works well for DNS because a query is self-contained. It works for TLS termination because the handshake completes at whichever instance receives it. It works for cache fills because a miss at one pop can be fetched from origin or a parent tier. Push stateful workloads through it and you inherit the session affinity problem you thought you avoided. A shopping cart pinned to one instance is exactly the workload anycast cannot save: when the path shifts mid-session, the new instance has no idea who the user is.
The weakness is coarse control. BGP does not weight by capacity, current load, or business priority. A small pop and a large pop advertise the same prefix and receive traffic based on topology, not headroom. You can influence this with BGP communities and prepending, but those are blunt instruments compared with weighted DNS answers. Prepending a few extra AS hops to a large pop to shed traffic is the routing equivalent of shimming a rack with washers: it works, it is fragile, and it surprises the next person who reads the config.
DNS steering buys precision at a cost
EDNS Client Subnet improves resolver-to-user accuracy by passing a truncated client subnet to authoritative servers. That lets DNS steering return an answer closer to the actual user rather than the resolver's location. Weighted answers enable gradual canary rollouts, and policy can steer by capacity, region, or provider. This is precision anycast cannot match.
The cost is latency in the control loop. TTL and resolver caching blunt fast failover. A record with a five-minute TTL means a degraded pop can receive traffic for five minutes after you remove it from rotation, and some resolvers ignore TTLs entirely. Stale caches can strand users on degraded pops long after your health checks have fired.
DNS steering also exposes topology to resolvers. That is useful for accuracy and awkward for privacy and operational secrecy. Anycast hides origin behind a single address. DNS steering publishes a map of where you have capacity, and that map is visible to anyone who queries it.
There is a quieter cost too: DNS answers are only as good as the resolver that asked. A large share of users sit behind resolvers that aggregate traffic from a wide geography, so the subnet hint is coarse and the answer is a compromise. You end up steering a metro-sized bucket of users with a single record, and the precision you paid for evaporates at the resolver boundary.
Where the two diverge under load
Anycast reroutes in seconds. DNS waits for TTL expiry. Under a sudden traffic spike, anycast shifts packets to the next nearest instance as soon as BGP reconverges, but it cannot steer by capacity. DNS can steer by capacity, but only after caches expire. Each plane solves the problem the other creates.
Anycast also hides origin topology, which simplifies DDoS absorption and reduces the attack surface. DNS steering exposes topology to resolvers, which is a feature for accuracy and a liability for operational secrecy. A hybrid pattern gets both: anycast as the front door for DNS, TLS, and stateless edge, with DNS steering layered behind it for regional weighting and canary rollouts.
The hybrid is not free. It adds a second control plane to instrument, and the two planes can disagree about which pops are healthy. That disagreement is the signal you want to alert on, not suppress.
Consider the failure ordering. During a regional incident, the anycast plane will move traffic within seconds, and the DNS plane will follow minutes later once caches turn over. If you have not rehearsed that sequence, the second shift lands on pops that are already absorbing the overflow from the first. The result is a cascade that looks like a capacity problem and is really a scheduling problem.
Health checks are the shared weak point
BGP withdraw is binary. Application health is graded. A pop can be 80 percent saturated while its health endpoint returns 200, and a DNS health check that polls every 30 seconds will not catch a spike that lasts 20. Mismatched thresholds between the two planes cause flapping, and flapping causes cache churn, which causes more flapping.
Instrument both planes against the same SLO. If the SLO is p99 latency under a threshold, both the BGP withdrawal policy and the DNS health check should derive from that metric, not from separate ICMP pings and HTTP status codes. A related piece on driver stack correctness burdens makes a similar point about layered systems disagreeing at the boundary.
Alert on divergence between BGP state and health state. If a prefix is advertised but the pop is failing its SLO, that is a page. If a prefix is withdrawn but the pop is healthy, that is also a page. The steady state is agreement, and divergence is the leading indicator of user impact.
Health check design deserves more care than it usually gets. A check that only proves the process is alive tells you nothing about whether it can serve. A check that exercises the full request path, including the dependencies that make the pop useful, costs more to run but catches the failures that actually hurt. The middle ground is to keep a cheap liveness probe for the orchestrator and a deeper readiness probe feeding the steering planes, with thresholds set from measured saturation curves rather than round numbers someone liked.
Instrumenting the handoff
The handoff between planes is where most hybrid deployments quietly rot. You need per-pop views of three things: what the routing plane believes, what the DNS plane believes, and what the service actually delivers. When those three agree, you have a boring day. When any two disagree, you have a lead on the next incident.
Build the comparison into your dashboards rather than your runbooks. A runbook step that says check whether BGP and DNS agree gets skipped at 3 a.m. A panel that colors the disagreement red does not. The cost is a small amount of pipeline work to join routing state with service metrics on a common pop key, and that join is the single most valuable piece of observability in an edge deployment.
What to actually deploy
Use anycast for DNS, TLS termination, and stateless edge services where session affinity is not a requirement. Reserve DNS steering for capacity-weighted rollouts and regional traffic engineering where precision matters more than failover speed. Set TTLs from a measured failover budget, not from habit or a default template.
Instrument both planes against the same SLO and alert on divergence between BGP state and health state. Test failover with real resolver cache behavior, including resolvers that ignore TTLs, before you trust a DNS-based failover in production. A related piece on checking maintainer signing keys makes the case for verifying assumptions at the boundary rather than trusting defaults.
Document which plane owns which failure mode, and rehearse the handoff. The hybrid pattern works when each plane knows what it is responsible for and what it is not. When they disagree, the instrumentation should tell you before your users do.