Search for a command to run...
Articles tagged with #incidents
The Folder Full of Stale Runbooks Every engineering org has a Confluence folder of incident runbooks. Every runbook was written during or after an incident. Each is a snapshot of how to fix one specific thing. After 2 years, the folder has 400 runboo...
Why Severity Is Broken at Most Companies Everyone has severity levels. Almost nobody agrees on what they mean. Ask ten engineers what SEV-2 means and you'll get eight different answers. This causes: Under-paged incidents (people thought SEV-3 meant ...
Everyone's Debugging, Nobody's Leading Five engineers in an incident channel. All debugging independently. Nobody coordinating. Three people checking the same dashboard. Two trying conflicting fixes. Customers waiting. This is what incidents look lik...
MTTR Is a Lagging Indicator Everyone tracks Mean Time to Resolve. Few understand what actually drives it. MTTR isn't one metric — it's four: MTTR = MTTD + MTTA + MTTI + MTTF MTTD: Mean Time to Detect (monitoring fired) MTTA: Mean Time to Acknowl...
Every Vendor Claims AI Magic Open any monitoring vendor's website and you'll see: "AI-powered incident detection!" "ML-driven root cause analysis!" "Intelligent alerting!" After evaluating a dozen AI ops tools and running three in production, here's ...
The Post-Mortem Nobody Learns From I've sat through hundreds of post-mortems. Most follow the same pattern: something breaks, someone writes a Google Doc, we have a meeting, we list action items, nobody follows up, the same thing happens again in 3 m...
