SERVERBROS

Runbook 03

An incident note that helps the next person

Capture the symptom, the timeline and the last change before the details disappear into a chat scroll.

A corridor between rows of server racks

Describe what a user can observe

“The website is broken” leaves too much room for interpretation. Record the hostname, affected path, first observed time and actual response. State whether the failure affects everyone you tested or only one connection. Distinguish an observation from a hypothesis: “the page returns 502” is evidence; “the database is down” needs its own check.

Build a short timeline

List the recent deployment, configuration change or alert that might matter, with a timezone. Keep entries factual and ordered. Include a successful check before the failure if one exists. Avoid filling gaps with confident guesses; an unknown start time is better than a fabricated exact one.

Record each action and its effect

For every change made during the incident, write what changed and what the same test showed afterwards. If the result did not improve, say so. This prevents the next operator from repeating an unsuccessful action or unknowingly reversing a useful one. Keep credentials and personal data out of shared incident notes.

Escalate with enough context

Provide the hosting support team with the affected service, the time window and a concise example of the failure. Explain what has already been checked. If traffic filtering is relevant, ask for the provider’s observations rather than treating a slow page as proof of an attack. Keep the support reference in the incident record.

Close with evidence of recovery

Repeat the original failing request and the important visitor journey. Confirm the intended release and any scheduled jobs or backups that were interrupted. Write a separate follow-up for prevention work. An incident is not the right moment to bundle several speculative improvements into an otherwise controlled recovery.

Put it into practice

Observed: /guide/ returns HTTP 502
First seen: 14:20 UTC
Last change: release at 14:12 UTC
Action: previous release restored at 14:28 UTC
Result: /guide/ returns 200; image loads
Open question: why did the new release fail?

Example only. These are not live service measurements.

Sources & further reading

Examples and worksheets on this page are illustrative. Documentation links were checked in November 2026; follow the relevant product’s current instructions.