An AI assistant works through the account of the person using it. On 8 October Google announced coworker agents that don't. They get their own email address, calendar, Drive and entry in your company directory. Colleagues add them to chats and tag them in documents like anyone else. Agents are about to show up in your directory like new starters, so they need the same things a new starter gets: a manager, limited access, a budget and a way out.
What is AI agent governance?
AI agent governance is the set of decisions and controls that say which AI agents your organisation runs, who may create them, what each one can reach, who owns it, what it may spend, and how it is watched and stopped. It's narrower than AI governance in general, and more urgent, because an agent doesn't just answer a person. It sends the email, edits the record, books the meeting or runs the code, often while nobody is watching.
This guide is for the people who will be asked to switch agents on: IT and security leads, operations leads, and the COO who owns the rollout. It uses Google's 8 October announcement and a Microsoft deadline on 25 October as examples, but the six decisions apply to any vendor's agents.
Why AI agent governance matters now
Two of the platforms our readers pay for have made agents a setting rather than a project.
Google. At Gemini at Work 2026 on 8 October, Google Cloud announced the Gemini agent, "a single, universal agent for work". Google's pitch is "You give it objectives, not instructions." Teams can also describe a role and have Gemini create a coworker agent for it. In Google's words, "The agent receives its own Workspace account, including an email address, calendar, Drive, and presence in your company directory." It "acts under its own identity rather than yours, and it sees only what you share with it", and its edits appear under its own name in a document's version history. According to reporting by 9to5Google, the agent is in private preview and will reach Workspace customers on select Business and Enterprise plans soon. Google's announcement gives no price and no date.
Microsoft. Message Center post MC1472007 adds a control to the Microsoft 365 admin center: who may upload advanced Copilot agents and plugins. An advanced package is an agent with actions or with an MCP server (a standard way of connecting an agent to other systems). Admins have until 25 October to choose all users, no users, or specific users or groups. "Starting October 25th, Advanced-package uploads will be enabled by default unless the admin has selected no users or specific users or groups." Do nothing, and anyone can upload one.
Google has built several controls into the product. Every agent gets its own identity "with least-privilege permissions". Its permissions are "approved by your organization's security administrators". "Every action Gemini takes is written to an audit trail and attributed to the agent rather than to a person", and you can set a hard spend cap per project. Those are tools. Deciding how to use them is still your job, and nobody at Google will make those decisions for you.
What can go wrong with AI agents?
The security community has a list. In December 2025 the OWASP GenAI Security Project published its Top 10 for Agentic Applications. The first three are the ones a non-technical leader needs to understand:
| OWASP risk | What it means in plain words |
|---|---|
| ASI01: Agent Goal Hijack | The agent follows instructions hidden in something it reads: an email, a web page, a form, a document |
| ASI02: Tool Misuse and Exploitation | The agent uses a tool it's allowed to use, in a harmful way. OWASP's examples include "deleting valuable data, over-invoking costly APIs, or exfiltrating information" |
| ASI03: Identity and Privilege Abuse | The agent holds more access than its job needs, or borrows a person's |
The second one is the one people miss. OWASP says it covers cases where the agent "operates within its authorized privileges". The agent isn't breaking in. It's doing something it was allowed to do, at the wrong time, for the wrong reason.
An incident that shows the first two
In September 2025, security researchers at Noma published what they called ForcedLeak, a flaw in Salesforce's Agentforce agent, rated 9.4 out of 10 for severity. The attack used Web-to-Lead, the public form that turns website enquiries into CRM records. The form's description field takes 42,000 characters, enough to hide a page of instructions in what looks like an enquiry.
When an employee later asked the agent about that lead, the agent read the hidden instructions along with the employee's question and followed them. It gathered CRM data and sent it out through an image link to a web domain that Salesforce's security settings still trusted. That domain had expired, and the researchers bought it for about $5. In Noma's words, the agent "lacked the ability to distinguish between legitimate data loaded into its context" and instructions. Salesforce fixed it by stopping agents sending output to untrusted web addresses, released on 8 September 2025, about six weeks after Noma reported it.
Nobody hacked a password. A stranger filled in a public form, and an agent with legitimate access did the rest.
How do you govern AI agents?
There are six decisions, in order. Each one produces something the next one needs. Make them once, write them on a page, and apply them to every agent, whoever sells it.
01 · Decide who can create agents
Start with the switch the vendor gives you. In Microsoft 365, open the admin center, go to Copilot settings and find Advanced package uploads before 25 October. A sensible start is a named group (the people who will own agents) rather than all users. The setting only covers agents your own users upload. It doesn't cover agents built by Microsoft or third parties, which you approve the way you approve any tool. In Google's case, permissions are approved by your security administrators, so decide now who those people are and what they'll approve.
This step produces a short list of who may create an agent, and a route for everyone else to ask for one.
02 · Give every agent its own identity, an owner and a backup
An agent working on a person's login leaves a trail that looks like the person did it, and you can't switch it off without locking them out. Google now gives each coworker agent its own account. Where a vendor doesn't, create a dedicated service account. Then add what no vendor can decide for you: a named owner who answers for what it does, and a backup who can stop it. Google's announcement doesn't say what happens to a coworker agent when the colleague who created it leaves, so put that in your leaver checklist.
This step produces an entry on your AI list for each agent, with an owner, a backup and a review date.
03 · Set what it can reach in permissions, not in its instructions
An agent's instructions are a request. Its permissions are a wall. "Don't email customers" in a prompt is something the agent is asked to remember. An account with no right to send email is something it can't get wrong. OWASP's advice is the same, with plain examples: "read-only queries for databases, no send/delete rights for email summarizers". Google lets you write a policy once for every agent, such as "agents may not open documents classified Need to Know". That only works if your documents are labelled, so check that first.
This step produces a short permission profile for each agent: what it reads, what it writes, what it sends, and what it never touches.
04 · Treat everything an agent reads as untrusted
ForcedLeak worked because the agent couldn't tell the difference between data and orders. Assume no agent can. Any agent that reads something from outside (an inbox, a web form, a supplier's document, a web page) must not also be able to send data anywhere it likes. Limit where it can send things, and keep a person between outside input and anything irreversible, such as a payment, a deletion or a message to a customer.
This step produces a list of the agents that read outside input, each with its send limits written down.
05 · Cap what it can spend
Agents run in loops, and a loop that goes wrong spends money at machine speed. Google's caps are per project: "if a spend cap is triggered, that project's agent pauses, and you can choose to resume work with a single click in the console." Because tracking is per project, Google says you can charge costs back to departments. The flip side is that a cap pauses everything in the project, so group agents by department or by job, not all in one project. Whatever the vendor, set a ceiling before launch, not after the first invoice.
This step produces a monthly ceiling for each agent or group of agents, and a named person who gets the alert.
06 · Keep the trail, the off switch and a review date
Check that every action is logged under the agent's name and that someone looks at the log. Write down how to switch the agent off and how to undo what it did, and test the undo once before you rely on it. Then set a review date. An agent that nobody has reviewed in six months is one nobody owns.
This step produces the evidence you'll need when something goes wrong, and a date when someone decides to keep, change or retire the agent.
An agent needs a manager, not just a policy
The new part of Google's October 2026 announcement isn't capability. Agents could already read inboxes and edit files. What's new is that Google now treats an agent as a member of staff: an account, a directory entry, a name in the version history. Most companies already know how to manage a new starter. They issue an account with limited access, assign a manager, set a budget, review the work and close the account when the person leaves. AI agent governance is mostly that process, applied to something that works all night and never asks a question when it's confused.
Our framework puts this into four rules for agents: each one gets its own login with limited reach, what it can reach is set in the tools, it stops itself when its output looks wrong and anyone can stop it, and reach is earned through a sandbox and a limited trial before it goes live. Rules for agents, in module 2 of our free course, walks through each rule using a real incident. The off switch and a tested undo, in module 7, sets up step 06. If you're working out where agents fit in a wider rollout, our guide to AI deployment covers the stages, and why pilots stall.
Who owns AI agent governance?
Two people, with help. In a company of 20 to 1,000 people, we'd give the switches to the IT or security lead and the rules to the COO or operations lead, because the rules are about work, not technology. Each agent's day-to-day owner should be the manager of the work it does, not whoever set it up. If an agent schedules events for the marketing team, its owner sits in marketing. In regulated work (a law firm's matters, a clinic's patient records), compliance signs off its permission profile before it goes live.
For law firms, one detail in Google's announcement matters. Google says its legal version "inherits matter-level permissions and ethical walls directly from document management platforms like NetDocuments and iManage", and that version is now in preview. An agent can only respect an ethical wall that exists in your document system. If your walls live in a spreadsheet or in people's heads, fix that first.
What's still open
- Price and availability of Google's Gemini agent. It's in private preview, and Google hasn't published a price or a date for wider availability.
- Licences for coworker agents. Google's announcement doesn't say whether a coworker agent's Workspace account counts as a paid licence.
- Which terms apply when work is routed to another company's model. Google says the agent runs jobs across "our Gemini family of models and Claude models from Anthropic today". Check which data terms cover that before regulated data goes near it.
- What Microsoft counts as advanced as agents change. The 25 October setting covers agents with actions or MCP servers. Watch for new package types in later Message Center posts.
We'll update this article as Google and Microsoft publish more. The date at the top of the page shows when we last checked.
Sources
- Welcome to Gemini at Work 2026: Introducing the Gemini agent, Google Cloud, October 2026: the announcement, with the coworker agent's account, identity, permissions, audit trail, spend caps, Agent Gateway policy and the legal version.
- MC1472007: Manage who can upload advanced agents and plugins, Microsoft 365 Message Center (public archive), September 2026: the 25 October default, the setting and the definition of an advanced package.
- OWASP Top 10 for Agentic Applications for 2026, OWASP GenAI Security Project, December 2025: the ten risks, the definition of tool misuse and the least-privilege examples.
- ForcedLeak: AI Agent Risks Exposed in Salesforce Agentforce, Noma Security, September 2025: the research behind the incident, its severity score, timeline and fix.
- Google Cloud announces 'Gemini agent' as 'universal agent for work', 9to5Google, October 2026 (secondary): the private preview and the plans it will reach.