Start with service objectives, not dashboards
At 02:13 on a winter morning, a Johannesburg payment service started paging its on-call engineer for rising API latency. The first alert was valid. The next 47 were not. A brief power interruption had moved traffic between availability…