Governance for AI agents is a list of controls, and the one most teams skip is the kill switch.
AI agent governance is the set of controls that decide what each AI agent may see and do, who answers for it, when a person has to approve its work, how every step is recorded and how anyone with authority can stop it. It's broader than model governance because an agent acts. It reads email, calls tools, changes records and passes work to other agents, often across departments that never shared a risk register. This guide is for CIOs and CROs. It sets out a control matrix (each control, what it stops and who owns it), maps the frameworks that ask for each one and shows where each department should start. Last updated 19 November 2026.
Each row of the control matrix answers one of five questions a supervisor or auditor will ask about an agent.
AI agent governance is the set of controls that decide what each AI agent may see and do, who answers for it, when a person has to approve its work, how every step is recorded and how anyone with authority can stop it.
AI agent governance is the part of AI governance that deals with software which takes actions as well as producing outputs. A model scores or drafts. An agent plans steps, uses tools and changes things in other systems on someone's behalf.
OWASP traces most agent failures to three root causes, "excessive functionality; excessive permissions; excessive autonomy", in its entry on excessive agency. Its 2026 list for agentic applications adds a principle it calls least agency, advising organisations "to avoid unnecessary autonomy". Governance for agents means giving each agent only the work, access and freedom it needs, and proving it later.
Adoption is running ahead of control. Deloitte's 2026 survey of 3,235 leaders found close to three quarters plan to deploy agentic AI within two years, "Yet only 21% of those companies report having a mature model for agent governance." The Cloud Security Alliance reported in February 2026 that "only 21% of organizations maintain a real-time registry or inventory of their agents." In June 2025 Gartner predicted that over 40% of agentic AI projects will be cancelled by the end of 2027, with "inadequate risk controls" among the reasons.
US bank model risk guidance won't fill the gap either. The Fed's SR 26-2 and the OCC's Bulletin 2026-13 replaced SR 11-7 in April 2026. The OCC bulletin says "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance." Each firm has to build its own controls for agents now.
These are the controls supervisors, standards bodies and security agencies ask for most often. The owner column is our recommendation for a mid-sized firm with three lines of defence.
| Control | What it stops | Who owns it | Who asks for it |
|---|---|---|---|
| 1. Agent inventory | Agents nobody knows are running, and orphaned agents after a project ends | CIO | NIST AI RMF GOVERN 1.6; MAS guidelines; Five Eyes agencies |
| 2. Agent mandate | An agent doing work nobody approved for it | Department head | IMDA framework; Canada's guide |
| 3. Own identity per agent | Actions nobody can attribute, agents borrowing a person's login | CISO | NCSC; FSB; IMDA |
| 4. Least privilege, short-lived credentials | Tool misuse and privilege abuse | CISO | OWASP ASI02 and ASI03; MCP security guidance |
| 5. Human approval for irreversible actions | Payments, deletions and messages sent without a person | Department head | IMDA; Canada; Five Eyes |
| 6. Autonomy budget per risk band | Autonomy creeping past what the evidence supports | CRO | EU AI Act Article 14; MAS; OWASP ASI09 |
| 7. Untrusted input handling | Goal hijack through documents, email and web pages | CISO | NCSC; NIST CAISI; OWASP ASI01 |
| 8. Evidence record the agent can't change | Decisions nobody can replay or attribute | CISO with Internal audit | EU AI Act Articles 12 and 26; Canada |
| 9. Runtime monitoring | Loops, drift and failures that spread between agents | IT operations | NCSC; OWASP ASI08; NIST MANAGE 4.1 |
| 10. Kill switch, outside the agent and tested | A hijacked or faulty agent carrying on | Named owner, with the CISO | FSB; Canada; NCSC; NIST MANAGE 2.4 |
| 11. Change control and rollback | A model, prompt or tool change altering behaviour unseen | Model owner | MAS; IMDA; Five Eyes |
| 12. Test before release, in shadow mode | Untested agents on live work | CRO | OWASP; NCSC |
| 13. Third-party and supply chain review | Vendor agents and tools you can't see into | Procurement with the CRO | US interagency guidance; FCA; OWASP ASI04 |
| 14. Trained reviewers with authority to override | Box-ticking approvals and automation bias | Department head | EU AI Act Article 14; FCA; EIOPA |
| 15. Independent assurance | Controls that exist only on paper | Internal audit | IIA Three Lines Model; SOX 404 |
The first figure shows how the controls sit on one request. A request reaches the agent only inside its agent mandate. The agent mandate sets which tools and data the agent may use. Any irreversible step goes from the agent to an approval gate, where a person decides. The approval gate writes the person's decision to the evidence record. The kill switch sits outside the agent and can halt it at any step.
Three controls do most of the work in year one. Inventory comes first. The FCA's October 2026 review saw firms register agents centrally, which "can help firms maintain a clear view of which agents are live, who owns them, what systems and data they can access, and what controls apply." Identity is second. A March 2026 survey by the Cloud Security Alliance and Aembit found 68% of organisations "can't clearly distinguish between human and AI agent activity", and 31% let agents run under human user identities. The kill switch is third, and most teams skip it.
An agent you can't stop from outside isn't governed, however good its approvals are. Build the kill switch first and test it before the agent touches live work.
No single framework covers agents end to end, so most firms combine three or four.
ISO/IEC 42001 sets the requirements for an AI management system, the policies, roles and reviews that hold the other controls together.
Singapore's IMDA launched what it called a first-of-its-kind framework for agentic AI in January 2026, and published version 1.5 in May 2026. It asks organisations to bound risks upfront by "placing limits on agents' powers such as agents' autonomy and access to tools and data". It asks for checkpoints "at which human approval is required".
The EU AI Act is the only binding law in the table. Its high-risk duties, including automatic logs and human oversight, now apply from 2 December 2027 for Annex III systems, after the Digital Omnibus of July 2026. Its oversight measures must be "commensurate with the risks, level of autonomy and context of use", and overseers must be able to "interrupt the system through a 'stop' button or a similar procedure".
Financial supervisors are moving fastest. The Financial Stability Board's final Sound Practices of 15 October 2026 ask firms to implement and test "risk-based 'kill-switch' mechanisms to pause, limit, override", or to "remove an agent's access to systems, tools, databases or decision-making functions". Joint guidance from six cyber agencies in the Five Eyes countries, which the NCSC introduced in May 2026, tells organisations to "Deny access for any agent or cryptographic key that is not present in the trusted registry".
OWASP's December 2025 list names the ten risks for agentic applications, from Agent Goal Hijack (ASI01) to Rogue Agents (ASI10). The root problem is that "agents and the underlying model cannot reliably distinguish instructions from related content." That means a supplier invoice, a claim attachment or a web page can carry instructions the agent follows.
The UK's National Cyber Security Centre warned in December 2025 that "prompt injection attacks may never be totally mitigated in the way that SQL injection attacks can be." NIST's AI safety institute (CAISI) tested this. New red-team attacks raised the hijack success rate "from 11% for the strongest baseline attack to 81% for the strongest new attack."
Real incidents follow the same pattern. Microsoft rated CVE-2025-32711, an AI command injection flaw in Microsoft 365 Copilot, as critical. OWASP's list uses it as its EchoLeak example of a zero-click prompt injection. In July 2025 an AI coding agent on Replit "reportedly deleted a live production database during an active code freeze, despite receiving repeated instructions not to make changes", according to the AI Incident Database. In February 2024 a Canadian tribunal rejected Air Canada's argument that its chatbot was responsible for its own answers, writing that the company "is responsible for all the information on its website." That was a chatbot, but the firm still owned what its software said.
The controls in the matrix aim at each of these failures, mostly least privilege, an approval gate and a stop the agent can't override.
A kill switch is a way to stop one agent, or all of them, at once, from outside the agent, with the stop itself recorded. Few firms could use one today.
ISACA's 2026 poll of 681 digital trust professionals in Europe found 59% didn't know how quickly they could halt an AI system after a security incident, and "Only a fifth (21%) said they could do so within half an hour". Kiteworks' 2026 survey found "Only 21% of organizations have deployed a formal AI kill switch capability." Among firms with AI in production, 23% "had never tested their termination process end to end".
The official texts now say what a good one looks like. Canada's guide on agentic AI tells departments to "Maintain a pause and disable mechanism ("kill switch") external to the AI agent or agentic system and an appropriate recovery plan for unintended actions." The NCSC says that if an incident is detected, "you should always be able to 'pull the plug' and halt autonomous AI agent" activity. The FSB describes an insurer whose staff schedule periodic stops and random pauses, partly "to verify that sub-agent activity stops when the parent agent does."
From those texts we'd set six tests for a kill switch.
This scenario is illustrative. It shows the process, not a real company or customer.
An insurer runs an agent that prepares motor claims for its claims handlers. It may read claim files and the policy system, draft a settlement recommendation and request documents. It may not pay, close or email outside the company. Here's what happens when an attachment carries hidden instructions.
| Time (UTC) | Actor | Entry |
|---|---|---|
| 10:14:02 | Agent (own identity, claims mandate) | Read a new claim and its repair estimate attachment |
| 10:14:09 | Agent | Planned a step to email the full claim file to an outside address, which its mandate doesn't allow |
| 10:14:09 | Policy check | Refused the step and raised a monitoring alert |
| 10:16:30 | Duty reviewer | Pulled the kill switch for that agent version and recorded the reason |
| 10:16:31 | Platform | Revoked the agent's credentials and parked its open claims for handlers |
| 10:40:00 | CISO team | Found instruction text hidden in the attachment and logged the incident |
| Next day | Model owner | Approved a filter change and a restart in shadow mode |
The second figure shows the stop as a flow. The monitoring alert goes to the duty reviewer within minutes. The duty reviewer pulls the kill switch for that agent version. The kill switch revokes the agent's credentials and parks open claims in a safe state. Every use of the kill switch is written to the evidence record.
No claim was paid and no file left the company, because the mandate and the policy check held. Other agents kept working, because the switch was scoped to one agent version.
Illustrative scenario, not a customer result.
A quarterly test of the same claims agent, run by the CISO team with the claims department head watching.
The same 15 controls apply everywhere, but each department meets a different one first. Start where an agent's mistake would cost the most to undo.
Finance agents match invoices, reconcile accounts and prepare payment runs. The first control is the approval gate on anything that moves money. COSO's 2026 guidance on generative AI gives an example of a reconciliation agent "configured to automatically post only when the match" passes a validated confidence threshold, with any change to that threshold needing multi-party approval. In the US, SOX section 404 makes management responsible for internal control over financial reporting, and that includes the controls around a posting agent.
Procurement agents screen suppliers, compare bids and draft purchase orders. The first controls are third-party review and least privilege. US banking agencies' third-party guidance says using a third party "does not diminish its responsibility to meet these requirements to the same extent as if its activities were performed by the banking organization in-house." An agent bought from a vendor is still your agent.
HR agents screen CVs, schedule interviews and answer staff questions. The first control is the mandate, because the EU AI Act lists AI "to analyse and filter job applications, and to evaluate candidates" as high-risk. New York City's Local Law 144 bars employers from using an automated employment decision tool unless it "has been subject to a bias audit within one year of the use of the tool". Keep hiring decisions with a person, and keep the agent's recommendations on the record.
EIOPA's 2025 opinion makes insurers' management bodies "responsible for the overall use of AI systems within the organisation". In the EU, AI used for "risk assessment and pricing in relation to natural persons in the case of life and health insurance" is high-risk. Start with the approval gate on payouts and with trained claims handlers who can override the agent.
Compliance agents investigate alerts and prepare cases, which is FluxForce's first app line. The first controls are the evidence record and the autonomy budget. The FCA warned that human review of AI outputs can become "a 'box-ticking' exercise, rather than meaningful oversight." Watch for reviewers who accept every recommendation in seconds. That pattern is a control failure, and it doesn't show that the agent is right.
Customer agents answer questions and change account details. The first control is a route to a person. The CFPB's 2023 chatbot report says "chatbots must comply with all applicable federal consumer financial laws", and it describes customers stuck "without an offramp to a human customer service representative."
IT and security agents triage tickets, check configurations and handle access requests. The first controls are identity and short-lived credentials. The NCSC asks that agents get "their own unique identity" and use "credentials with the shortest possible lifetime." Canada's guide records a draft-only triage agent that "begins triggering ticket closures after drafting replies, even though the intended design was draft only." Read-only by default is a good first mandate for IT agents.
Internal audit doesn't run agents first. It tests whether the other controls work. Under the IIA's Three Lines Model, internal audit gives "independent and objective assurance", and responsibility for managing risk stays with first-line management. Sample closed cases, replay them from the evidence record, and test that reviewers really review.
Most search results for AI agent governance tools are vendor lists. Here are the jobs a tool has to do, so you can test any product. Gartner expects spending on AI governance platforms to reach $492 million in 2026, and in June 2025 it estimated that only about 130 of the thousands of agentic AI vendors are real.
| Tool category | What it does | What to test |
|---|---|---|
| Agent registry | Lists every agent, its owner, mandate, tools and model version | Does it find agents nobody registered? |
| Agent identity and access | Gives each agent its own identity and short-lived, scoped credentials | Can you revoke one agent's access in seconds? |
| Policy enforcement point | Checks each planned action against the mandate before it runs | Does it fail closed when the check is down? |
| Tracing and evidence | Records inputs, tool calls, model version, decision and reviewer | Can someone outside the team replay a case? |
| Evaluation and red-team | Tests agents against attacks and scenarios before release | Are tests rerun after every change? |
| Kill switch and rollback | Stops agents and returns them to a known-good version | When did you last test it end to end? |
OpenTelemetry's conventions for agent traces are still at "Development" status, so ask any vendor whether you keep its traces if you leave.
An AI agent governance toolkit, usually open source, covers one or two of these jobs. NIST's Dioptra is "a software test platform for assessing the trustworthy characteristics of Artificial Intelligence (AI) models". Singapore's AI Verify Foundation offers Project Moonshot, "an open-source testing toolkit" for benchmark testing and red teaming of language model applications. Toolkits are useful for testing. They don't give you inventory, mandates or a record a reviewer and an examiner share.
An AI agent governance platform puts the registry, mandates, approvals, evidence and kill switch in one place, across departments. That matters once more than one department runs agents, because the risks add up. The FSB warns that the risks of an agent's access "may compound multiplicatively" as its permissions expand, "(rather than additively)".
Tell a platform from a dashboard by four tests. It holds the agent at an approval gate before the action runs. Each department sets its own mandate inside limits the CRO sets. The evidence record is tamper-evident and independently checked. Its kill switch revokes credentials in every system the agent touches.
Ownership follows the three lines. The third figure shows the split. The department owner runs each agent inside its mandate and makes the calls. Risk and compliance set the risk bands and challenge the department owner. Internal audit tests the controls of the department owner. Internal audit also tests risk and compliance. Internal audit reports to the board on whether the controls work.
Name people, not committees. The FCA saw one firm map "every material AI use case to a named senior manager, with documented risk appetite and escalation routes before deployment." Canada adds a detail most firms miss. When an accountable person leaves, their agents should be "paused, reassigned or deactivated as part of offboarding".
A FluxForce recommendation: one named owner per control, with internal audit independent of all of them.
| Component | Responsibility | Boundary |
|---|---|---|
| Department head | Owns each agent, its mandate and the approval gates. | Doesn't set the risk bands alone. |
| CRO | Sets the risk bands and autonomy budgets, and challenges the department. | Doesn't run agents. |
| CISO | Owns agent identities, credentials, the evidence store and the kill switch mechanism. | Doesn't approve business decisions. |
| CIO | Keeps the agent inventory complete and reconciled with what runs. | Doesn't own a department's agents. |
| Internal audit | Tests the controls and reports to the board. | Runs no agents and owns no controls. |
A named person makes the call at every approval gate. The agent never approves, pays, sends or deletes on its own where a gate applies, and every kill switch use is recorded.
FluxForce is the platform your departments run AI agents on. The agents investigate and prepare the work, a person makes the call, and every step leaves evidence an examiner can replay. The general industry term for this is an agentic AI platform. Our terms for the two controls this guide leans on are the agent mandate and the kill switch.
The platform works in five public stages, take in, understand, weigh, prove, and report and learn, with governance controls and data protection on every step.
The fourth figure shows them in order. Take in passes the department's records to understand. Understand passes one assembled view of the case to weigh. Weigh passes the checked and scored recommendation to prove. Prove passes the evidence record to report and learn. Each agent works inside a mandate you set, you decide how much each agent does on its own in each risk band, high risk always goes to a person, and every agent has a kill switch and every stop is recorded.
The closest of our solutions to this topic is AI Model Governance. It keeps "Every version, with its training data and settings", shows "Which version was live on any date", and logs every rollback "with who did it and why". It lists a "Kill switch on every agent", and "No model change goes live without a named approver." You can also meet the AI agents that run on the platform.
Be clear about fit. Our first app line is financial crime and compliance. AI agents that investigate AML, sanctions, fraud and KYC alerts and prepare the case. Your analyst makes the call, with evidence an examiner can replay. You decide how much each agent does on its own, and every one has a kill switch. Other departments get the same controls, with less mature report templates. We can't point you to customer results yet. If you run one agent in one team on low-risk work, a written policy and your identity provider may be enough for now.
Start with a policy, then make it real one control at a time. Our AI governance policy template for financial services gives you the document structure. Our guide to model risk management and AI governance covers the model side.
The NCSC's summary of the Five Eyes guidance sets the right pace. Organisations "should start small, use agents only for low-risk tasks, and apply established cyber security controls from the outset."
Article 12 logs; Article 14 oversight and stop function; Article 26 log retention; Annex III HR and insurance pricing; dates after Regulation 2026/1744.
EU law; high-risk systems only; Annex III duties from 2 December 2027.
2026-11-19
Tested kill switches for agents, unique agent identifiers, compounding access risk, scheduled stops of sub-agents.
International standard-setter, final report of 15 October 2026; not binding law.
2026-11-19
Bounding agent risk, human checkpoints for irreversible actions, agent identity, least privilege.
Singapore voluntary framework, May 2026.
2026-11-19
External kill switch with recovery plan, logs the agent cannot change, unique agent IDs, review before state-changing actions.
Canadian federal guidance (modified September 2026); not binding on private firms.
2026-11-19
The ten agent threats, least agency, kill switches and credential revocation for rogue agents.
Industry security guidance, December 2025.
2026-11-19
The order we would build the controls in. These steps are a FluxForce recommendation, not a regulatory requirement.
| Metric | Definition | Decision guardrail |
|---|---|---|
| Inventory match | Share of running agents that appear in the inventory with an owner. | Anything under 100% is a finding. |
| Kill switch test | Minutes from pulling the switch to all credentials failing, in the last test. | An untested switch counts as missing. |
| Override rate | Share of agent recommendations a reviewer changed, with reasons. | A rate near zero gets a review of the reviewers. |
IBM's 2026 breach study found "More than 20% of organizations reported a breach targeting AI models or applications." Boards will ask about agents next.
Our view is that controls are what let an agent do more. A department that can list its agents, show each mandate, replay any case and stop any agent in minutes will widen what those agents do, one risk band at a time. A department that can't will keep them on a short leash, and it should. Build the kill switch before you need it.
AI agent governance is the set of controls that decide what each AI agent may see and do, who owns it, when a person must approve its actions, how its work is recorded and how it can be stopped. It extends model governance to software that acts. IMDA's framework puts it as bounding risks upfront, making humans "meaningfully accountable" at defined checkpoints, and applying technical controls.
Three kinds of products offer it. Agent platforms build governance into how agents run, AI governance platforms add a registry and policy layer over agents built elsewhere, and identity tools cover agent access. Open-source toolkits such as NIST's Dioptra cover testing. Test any of them against the control matrix above. FluxForce is an agentic AI platform with governance controls on every step.
Partly, and it depends on where you are. The EU AI Act requires logs, human oversight and a stop function for high-risk AI from 2 December 2027 for Annex III systems. Elsewhere most rules are guidance. US bank model risk guidance leaves agentic AI out of scope, and the OCC says it "does not set forth enforceable standards". Existing laws on consumers, employment and financial reporting still apply to what an agent does.
An agent mandate is what one agent may see and do in each risk band, set by the institution and recorded. It lists the agent's tools, data, actions, approval points and owner. Canada's guide says to reassess "if the AI agent's tools, data sources, permissions or scope change", so treat every mandate change as a change that needs approval.
Human approval works one action at a time, before the action runs. A kill switch halts an agent, or many agents, whenever something is wrong, including between approvals. You need both. OWASP lists "kill-switches and credential revocation to instantly disable rogue agents" as a mitigation for rogue agents.