Designing for the edge: latency as a feature

Latency is the only metric users feel directly. A 200 ms API call is fine on a dashboard, but unacceptable when someone is tapping a payment button at a train station.

Edge computing lets us move logic closer to users, but it also changes how we think about state. Caching, consistency, and failover all need to be re-evaluated when your runtime is distributed across dozens of locations.

The teams that do this well treat latency as a feature. They measure it per region, per device, and per user journey. They know that shaving 100 ms in the right place often matters more than shaving a second in the wrong one.

In this post I walk through a few patterns I've used: stale-while-revalidate at the edge, partial hydration of API responses, and graceful degradation when the closest node is unhealthy.

Request path

The edge answers most reads directly, and only falls through to origin when it must.