Skip to content

Observability

The property of a system that lets you work out what it is doing from the outside, using the signals it emits, without adding new instrumentation first.

Monitoring answers questions you thought of in advance: is the disk full, is the error rate above two percent. Observability is the broader property that lets you answer a question you did not anticipate, such as why orders from one specific customer stopped syncing on Tuesday, using data the system already produces.

For automation and integration work, the practical signals are these:

  • Structured logs. Key and value fields, not sentences. A line of workflow, order_id, status and code fields is searchable. “Failed to push invoice” is not.
  • Counts at every stage. How many records came in, how many went out. Silent data loss shows up as a mismatch between the two long before a human notices missing output.
  • A heartbeat. A durable timestamp written at the end of each successful run, plus a separate check that reads it and alerts when it is stale. This is the only signal that survives the workflow not running at all.
  • A correlation id. One identifier attached to a unit of work and carried through every hop, so a single order can be traced across four systems.
  • Queue depth and age. Depth tells you about load. Age tells you about a consumer that has stopped, which depth alone can hide.

The test of whether a system is observable is not how many dashboards it has. It is whether you can answer, right now and without deploying anything, the question: did the thing that should have happened, happen?

If the honest answer is “someone would notice eventually”, the system is not observable, and it is one changed credential away from a silent failure that lasts weeks. The signals also have a second audience: they are what a runbook points at when it tells a tired person how to confirm which failure they are looking at.