Why Most Post-Mortems Fail
Post-mortems get a bad reputation because they're often written to satisfy a checkbox, not to drive change. The report gets filed, the action items rot in a backlog, and the next incident looks suspiciously familiar.
The goal isn't documentation for its own sake. It's to understand what your system revealed about itself under stress—and to make it harder for that failure to repeat.
Blameless Doesn't Mean Accountability-Free
Blameless means you assume engineers made reasonable decisions given what they knew at the time. It does not mean no one owns the follow-up work.
When you focus on people as the cause, you stop looking at the systems, processes, and incentives that made the failure possible. Blame also discourages honest reporting, which is exactly the opposite of what you need.
The shift: instead of "who broke it," ask "what conditions made this breakage possible and likely?"
Before You Write Anything: Gather the Raw Data
A post-mortem is only as good as its timeline. Before anyone starts drafting, collect:
- Monitoring and alerting timestamps (when did the first signal fire?)
- Deployment and config-change logs
- On-call notes and chat history from the incident channel
- Customer-facing impact data: error rates, latency percentiles, affected regions
- Detection lag—the gap between when the problem started and when your team knew about it
If you're running multi-region uptime monitoring, pull the check logs for each region. They'll often show you where degradation appeared first and whether it was a localised failure or a global event. That geographic data changes your root-cause analysis.
The Structure That Actually Works
1. Incident summary
Two or three sentences. Date, duration, user-visible impact, severity. No analysis yet—just facts.
2. Timeline
Chronological, to the minute where possible. Mark the moment the problem began (even if you only know this in hindsight), the moment it was detected, the moment it was escalated, and the moment it was resolved. The gap between "began" and "detected" is almost always where the most valuable learning lives.
3. Root cause analysis
Use the five-whys or a simple cause-and-effect chain. Keep going until you hit a systemic factor—a missing test, an undocumented dependency, an alert threshold that was never tuned—not a human error.
Example chain:
- Service returned 503s → database connection pool exhausted → pool limit set during initial deploy, never revisited → no alert on pool utilisation → no runbook for this failure mode
That last item is where you act.
4. Contributing factors
List things that made the incident worse or harder to resolve, even if they weren't the root cause. Ambiguous runbooks, missing dashboard panels, an alert that cried wolf too often and got muted—these belong here.
5. What went well
This section gets skipped and it shouldn't. If your on-call engineer caught an anomaly early because of a well-tuned alert, write that down. You want to reinforce the behaviours and tooling that helped.
6. Action items
This is where most post-mortems die. Keep action items useful with this checklist:
- Each item has a single owner (a person, not a team)
- Each item has a due date
- Each item is specific enough that you'd know when it's done
- At least one item addresses detection or alerting, not just the fix
- Items are tracked somewhere your team actually looks (not just the doc)
Vague items like "improve monitoring" or "add more tests" don't get done. "Add a Pingy alert on /api/health with a 30-second interval for all three production regions, by Friday" does.
Running the Post-Mortem Meeting
Keep it to 30–45 minutes. The document should be drafted before the meeting so you're discussing and refining, not writing from scratch together.
Assign a facilitator who wasn't the primary responder. Their job is to keep the conversation on systems and processes, redirect any blame that surfaces, and make sure every action item gets an owner before the call ends.
Invite everyone who was in the incident channel. Leave the meeting open to observers. The more people who understand what happened, the better.
After the Meeting
Share the final document with your broader engineering team, not just the people involved. A failure in your payments service might contain lessons for the team running your notification pipeline.
Schedule a quick follow-up in 30 days to check action item status. If items are slipping, that's a signal about prioritisation and capacity—worth a separate conversation.
Key Takeaways
- Blameless means focusing on systems, not protecting people from accountability
- The detection gap—time between failure start and alert—is often the most actionable metric
- A timeline built from real logs beats recollection every time
- Action items without owners and due dates are wishes, not commitments
- Share post-mortems widely; the learning shouldn't stay inside one team
- Review open action items 30 days later, or they won't get done