Monitoring goes wrong in a specific way. Someone installs a tool, enables the defaults, and the team receives forty notifications in the first week — none of which required action. Within a month the channel is muted, and the one alert that mattered, usually a website down alert, arrives into a room where nobody is listening.

The short list worth alerting on

Does the site answer? From outside your own network, every minute. The single most valuable check and often the only one a small product truly needs.

Are errors climbing? Not any error — a rate above normal. One failed request is life; two hundred in a minute is an incident.

Has background work stopped? Queues, scheduled jobs, imports. These fail silently by nature, and the failure is usually discovered by a customer asking where something is.

Are the business numbers moving? Orders per hour, sign-ups, payments. When the technical checks are green and this is flat, something is broken in a way no infrastructure alert will catch. It is the most valuable alert and the least often configured.

Is anything about to run out? Disk, certificates, API quotas. Predictable, preventable, and embarrassing when missed.

What belongs on a dashboard, not in an alert

CPU and memory graphs. Response time percentiles. Deployment history. All genuinely useful when you are investigating something, and none of them worth waking a person for on their own — high CPU is not a problem, it is a fact.

The rule that keeps it useful

Every alert must have an action attached. If the honest answer to "what would I do at 3am when this fires" is "look at it in the morning", it is not an alert; it is a note. Move it to a dashboard, keep the channel quiet, and preserve the meaning of the ones that remain.

And the part people skip: testing the website down alert

Test it. Break something deliberately in staging and confirm the alert arrives, in the channel you expect, with enough information to act. Monitoring that has never fired is indistinguishable from monitoring that does not work, and you will find out which on the worst possible night. Alerts that have been fired on purpose, at least once, are part of the monitoring we hand over.

The question before this one: do you actually need Kubernetes?.