There is a rollback mechanism in my system. It is designed, it is staged, and it has no permission to act. That was not an oversight. It is the whole point.
This is the third piece in a series about claims. The first was about documentation: sentences we write about a system that quietly stop being true. The second was about confidence: the claims a system makes about itself, almost none of which are ever graded. This one is about the layer underneath both, and the one I think is being skipped fastest right now. What is the system allowed to do.
Everyone is building agents that act. Very few people have written down what those agents are permitted to decide.
Three kinds of permission
When I look at my own systems, permission separates into three questions, and conflating them is where most of the trouble starts.
What is the system allowed to compute. In my football analysis product, team names, player names, tournament identities and statistics never come from model output. They are injected from verified data after generation and the model is instructed to treat them as fixed. The reason is empirical rather than philosophical: the model biases structurally toward famous clubs and invents plausible but wrong entities. So identity is treated as data, not as something the model may decide. The rule I keep returning to is that a model is not allowed to compute what can be checked.
What is the system allowed to claim. That was the last article. Every displayed confidence is a falsifiable prediction, and if you are not grading it you are serving a number you have no basis for.
What is the system allowed to do. This is the one that matters now, because generation got cheap and action did not get safer at the same rate.
The rollback
My system grades its own predictions weekly against real outcomes. Hit-rate floors are defined for the overall and free-tier paths. When one is breached, an alarm fires.
Then a human opens a runbook and performs the rollback by hand.
Detection is automated. The response is not. That distinction is documented rather than blurred, and I write it that way in every document because the blurred version is the one that spreads. It is very easy to describe a system as rolling itself back when what you actually have is an automatic alarm and a person who knows what to do about it. I made exactly that error in my own documentation and found it during an audit.
The automated version exists. It is designed and staged, and stage one deliberately holds no permission to act. The rule does not get permission until it has been back-tested against the full grading history.
A rollback rule that has never been tested against its own past is a guess with permissions.
The case that proves it
I would find that argument abstract if I had not watched it play out.
A calibration alarm fired on my system out of season. Around eighty graded events in the window, ten of them in the high-confidence bucket. Ten events tell you almost nothing. I chose not to refit. Two weeks later, at roughly four hundred events, calibration measured as good.
Now run that same week with an automated responder holding permission to act. The alarm fires, the rule executes, the model refits on ten events, and the system learns the shape of one quiet week instead of the shape of the domain. Nobody is negligent in that story. The automation does exactly what it was built to do, promptly and correctly, on a signal that meant nothing.
The rule that came out of it is to verify sample size before acting, and it is now a hard constraint in the automated design rather than a habit I remember to follow. But notice what actually protected the system that week. Not the rule, which did not exist yet. What protected it was that nothing had permission to act without a human in the path.
Permission is earned by evidence, and the evidence has to come from somewhere. Usually it comes from the period during which you did the thing manually and watched what happened.
Reversibility is part of permission
There is a second half to this that is easy to miss.
When I found two modules producing measured overconfidence, I did not delete them. I put them behind a flag with a documented closed state and made a deterministic calculation the default.
The distinction is not cosmetic. Deletion is irreversible. A flag reverses without a deploy. And that flag is now the lever the staged automated response uses, which means the decision to switch something off and the mechanism to switch it back on are the same object.
An action a system is permitted to take should be an action that can be undone at the speed it was taken. If reversing costs a deploy, a review and an afternoon, then the automation is not really safe, it is just fast in one direction.
The same logic sits in how guards fail in my system. Guard failures fail open on the user path and fail closed on security and quota. A guard error should never silently deny a paying customer, and an authentication error must never grant free access. Those are two opposite defaults, chosen per surface, and writing down which is which is most of the work.
What I would ask
The industry conversation right now is about capability. Can the agent call the tool, chain the steps, complete the task. Those are answerable and increasingly the answer is yes.
The questions I would ask before shipping one are duller and, I think, more load-bearing. What is this allowed to decide on its own. What evidence gave it that permission. How fast can the action be undone, and by whom. And what happens when the signal it acts on turns out to mean nothing.
If those answers are not written down anywhere, the system does not have permission. It just has access.