Runbook: policy rollback

A policy activation broke things. Revert narrowly, not by dropping enforcement.

Trigger. A policy was activated and agents are now being refused things they should be allowed to do, or — worse — allowed things they should not.

#First, confirm it is the policy

bash
veefore logs --decision BLOCK --limit 50

Read the explanation on a representative refusal. It names the policy, the version, and the deciding rule.

bash
curl -s $CP/v1/actions/act_01ABC -H "Authorization: Bearer $KEY"

Not every refusal is a policy refusal

AUTHORITY_EXCEEDED is delegation, not policy. TOOL_DISABLED is a tool being off. CAPABILITY_NOT_GRANTED is a missing grant. ENVIRONMENT_PAUSED is someone else's containment.

Confirm the deciding rule belongs to the version you just activated before rolling anything back. Rolling back the wrong thing while the real cause continues is a worse position than where you started.

Do not drop the enforcement mode

It is faster to type and it switches off everything, including the rules that were working. Roll back the policy — it is narrower and it is reversible in the same way.

#Find out exactly what changed

bash
veefore policies diff pol_01ABC --from 4 --to 5

A structured diff between versions: rules added, removed, re-prioritised, conditions changed, the default decision changed.

Check the default decision first

A defaultDecision changed from ALLOW to BLOCK on a policy bound at ENVIRONMENT scope denies everything in that environment — because strictness wins over priority, so every other policy's ALLOW loses to it.

The symptom is "nothing works and no rule looks relevant", because the deciding rule is reported as Policy default rather than as something somebody wrote. This is the single most likely cause of a broad breakage after an activation.

#Roll back

bash
veefore policies deploy pol_01ABC --version 4 --reason "Rolling back CHG-4411; version 5 default decision broke unrelated agents"

Rollback creates a new version with the old content rather than moving a pointer backwards. Version 6 is version 4's content. History stays linear, and the audit trail reads as a sequence of decisions rather than a mutable state you have to reconstruct.

Requires policy:rollback — high impact, step-up gated — plus a reason.

#Confirm it worked

bash
curl -X POST $CP/v1/policies/simulate \
  -H "Authorization: Bearer $KEY" -H 'content-type: application/json' \
  -d '{ "agentId": "agt_01JAV…", "tool": "refund_customer", "amount": 50000 }'

Simulate the specific shape that was failing. Side-effect free, and it tells you whether the rollback actually addressed it before you wait to find out from traffic.

Then watch:

bash
veefore logs --follow --decision BLOCK

#Then fix it properly

Rollback buys you time. It does not fix the policy.

Reproduce it in a test case

bash
veefore policies test

Add a Given/Then case for the shape that broke. If the broken version had passed this case, it would never have activated — activation refuses while any case fails.

This step is the whole point. A rollback without a new test case means the same mistake can be made again by the same route.

Fix the rule

Common causes, roughly in order of frequency:

Cause	Fix
`defaultDecision` changed on a broadly bound policy	Bind narrower, or revert the default
Amount off by 100 — minor units	`100000` is ₹1,000.00, not ₹100,000
`gt` where `gte` was meant, or the reverse	Test the boundary values, not the middle
A `BLOCK` rule wider than intended	Narrow the condition. Do not try to out-prioritise it
A new rule interacting with an existing one	Read `explanation.matched_rules` — every rule that matched is listed

Simulate against real traffic

veefore deploy preflight replays your recent observed traffic and shows exactly what would change. That is a far better signal than a handful of cases you thought of.

Reactivate

bash
veefore policies deploy pol_01ABC --version 7 --reason "CHG-4411 retry; default decision scoped to the agent, regression test added"

#If the problem is the opposite

A policy activation that allowed something it should not is more urgent, and the order is different: contain first, then roll back.

bash
# Stop the thing that is happening.
curl -X POST $CP/v1/tools/tool_01ABC/disable \
  -H "Authorization: Bearer $KEY" -H 'content-type: application/json' \
  -d '{ "reason": "Policy v5 permitted unapproved refunds; containing while rolling back" }'

Then roll back, then re-enable the tool. Then open an incident and work out what got through — the action records tell you exactly which requests were permitted and what they did.

#Prevention

Boundary test cases. 100000 and 100001, not 50000. Off-by-one at a threshold is the most common policy bug.

Bind deny-by-default narrowly. AGENT or TOOL scope, not ENVIRONMENT.

Diff before activating. veefore policies diff takes two seconds.

Use the preflight replay. It is the only check that uses your real traffic.

One change per activation. Three rules changed at once means three candidates when something breaks.

Policy

Precedence, and why strictness beats priority.

Write your first policy

Boundary test cases.

Gate your pipeline

Catching it before activation.

Policy API

Diff, simulate, activate, rollback.