Guardrails

Guardrails are checks that sit around a model. Some inspect what goes in, such as personal data that should be removed or instructions that should not be followed. Some inspect what comes out, such as an answer that makes a claim the sources do not support. Others inspect what the agent is about to do, such as a write to a system it should not touch.

Guardrails are part of the harness. They do not make a model correct. They make certain kinds of failure less likely to reach a person or a system.

In practice

I place guardrails at three points: before the model sees an input, before an output is shown or stored, and before any tool call with side effects runs. Each guardrail has a clear result: block, flag for review or log only. A guardrail that only logs must have a named person who reads the log.

Guardrails are tested as well. A set of known bad inputs and known bad actions is kept with the evaluation set, and the guardrails must catch them on every release.

Example

[Composite] A customer-support assistant was given a guardrail that rejected any answer quoting a price. The rule was too broad and blocked correct answers about published plan names. The team changed the guardrail to check against the price list, which was the actual risk, and added the plan-name cases to the test set.

Common mistakes

• Relying on the model's own judgement as the only guardrail.
• Writing guardrails that block everything, then removing them under pressure.
• Logging a guardrail hit with nobody assigned to read it.