AI agents are failing at office jobs — and the reason is not what most people assume. The models are not too dumb. The problem is that agents show up to work knowing nothing about the company they are supposed to serve.
In July 2026, the New York Times ran an interactive experiment that deployed AI agents to perform real office tasks. The results were mixed in ways that matter. Agents could handle some things, but broke down on others — not because reasoning failed, but because the agents had no access to the institutional knowledge that any human employee would absorb in their first week on the job.
That gap has a name: missing business context. And until it gets solved, AI agents at office tasks will keep underperforming in exactly the ways the NYT experiment exposed.
What the NYT experiment actually found
The Times tested AI agents against a range of real office work — the kind of tasks that fill the days of the millions of back-office workers identified in a companion piece as most exposed to AI displacement: HR, billing, payroll, scheduling, and coordination work.
The agents performed some tasks capably. But they stumbled on anything that required knowing this company's processes, this team's conventions, or this customer's history. Generic intelligence is not the same as contextual competence.
This tracks closely with what CMU researchers found when they built TheAgentCompany — a simulated office environment designed to benchmark AI agents on real-world tasks. Their findings were stark: most AI agents would make terrible office workers. Agents got lost trying to find files, invented fake coworkers when they needed information, and failed to complete routine tasks like writing performance reviews — not because they lacked language ability, but because they lacked situational awareness. One agent could not close a pop-up window blocking a website it needed to use. Another simply did the task wrong because it did not know the relevant internal policy.
The pattern across both studies is consistent: AI agents at office tasks fail at the boundaries of their context, not at the limits of their intelligence.
Why context is the real bottleneck
A new hire who knows nothing about your company is still useful on day one because you can hand them a document, point them to the right Slack channel, or sit them next to someone who knows the answer. AI agents, in their current default state, get none of that onboarding. They arrive with world knowledge and no company knowledge.
Consider what a competent office worker actually knows that an AI agent typically does not:
- Who owns which accounts and what was promised to each
- Where the relevant files live and what naming conventions the team uses
- Which processes are documented and which exist only in someone's head or in old Slack threads
- What the current state of a project is, based on last week's meeting notes
- Which vendors are on hold, which invoices are disputed, and why
None of this is in the model's training data. It lives in your Slack, your Gmail, your Notion, your CRM, your accounting system. An agent without access to that knowledge will hallucinate answers, get stuck, or complete tasks incorrectly — exactly what both the NYT experiment and the CMU benchmark observed.
As Unitary's guide to back-office AI automation notes, AI agents can hallucinate and misinterpret context — and the key to getting reliable results is deploying them inside a structured framework where they have access to the right information and can escalate when they do not.
The three failure modes the experiments revealed
Looking across the NYT experiment and the CMU benchmark, three distinct failure modes emerge for AI agents doing office work:
1. Information retrieval failures
Agents could not find the right file, the right thread, or the right record — because they did not know where to look and had no index of the company's actual information landscape. They either gave up or invented an answer.
2. Policy and convention failures
Agents completed tasks in ways that were technically coherent but wrong for this company. They did not know the internal approval workflow, the preferred format for a deliverable, or the exception that applies to a specific client.
3. Relationship and history failures
Agents had no memory of prior interactions. They could not answer "what did we tell Acme last month" or "what is the status of the Henderson contract" because that history lived in email threads and CRM notes they could not see.
All three failures share a root cause: the agent was operating without a company knowledge base.
What a business context layer actually fixes
The solution is not a smarter model. It is giving agents access to the right information before they start the task.
This is what a business context layer for AI does. Instead of leaving agents to work from world knowledge alone, a context layer ingests your company's data from the apps you already use — Slack, Gmail, Notion, Google Drive, HubSpot, Salesforce, QuickBooks — and makes it available to the agent at query time, with source citations and permissions intact.
The difference in practice:
| Without context | With business context layer |
|---|---|
| Agent invents a coworker's email address | Agent pulls the correct contact from your directory |
| Agent writes a generic performance review | Agent references the employee's actual project history |
| Agent cannot find the relevant file | Agent retrieves the correct document with a source link |
| Agent applies the wrong policy | Agent reads your internal policy doc and applies it correctly |
| Agent has no history with a client | Agent reads the CRM notes and prior email thread |
The CMU researchers' agents failed because TheAgentCompany's internal knowledge was not accessible to them. The NYT agents hit the same wall. The fix is not a new model release — it is an architecture change.
This is also why RAG alone is not enough for most business use cases. A retrieval-augmented pipeline can fetch documents, but it does not give you permissioned, cross-app, source-cited context that stays current as your business changes. A purpose-built context layer does.
How to actually give AI agents your company's context
The practical path forward for teams that want AI agents to work reliably on office tasks:
1. Identify where your institutional knowledge actually lives. For most companies it is split across at least four or five systems: email, a messaging tool, a project management tool, a CRM, and a file store. Any context layer needs to reach all of them.
2. Set permissions before you connect. Not every agent should see everything. Customer contract details, HR records, and financial data need to be scoped correctly from the start — not retrofitted later.
3. Expose context through a protocol agents can consume. The Model Context Protocol (MCP) is the emerging standard for this. An MCP server exposes your company's knowledge in a form that AI agents — Claude, ChatGPT, Cursor, Codex — can query directly, without you building a custom integration for each tool.
4. Keep it current. Static snapshots go stale. The context layer needs to reflect the state of your business today, not the state it was in when someone last ran an export.
5. Require source citations. An agent that tells you "the Henderson contract expires in March" is only useful if you can verify where it got that. Source-cited answers are auditable; hallucinated ones are not.
Gyld handles all five of these steps as a single product. It ingests from the apps your team already uses, permissions the knowledge at the individual, team, or company level, and exposes it as MCP servers that any AI agent can plug into — no pipeline to build, no fine-tuning required. When an agent answers a question, it cites the source document.
Takeaways
- AI agents fail at office tasks primarily because they lack company-specific context, not because the underlying models are weak — a finding consistent across both the NYT experiment and the CMU TheAgentCompany benchmark.
- The three failure modes are information retrieval, policy/convention gaps, and missing relationship history — all solvable with a proper business context layer.
- The right architecture exposes company knowledge to agents through MCP servers, with permissions and source citations, so agents can answer questions about your actual business instead of hallucinating answers.
If you want to see how a context layer changes what AI agents can do with your company's data, start building your company brain with Gyld — connect your first app and ask a question your agents could not have answered yesterday.
Frequently asked questions
Why did the NYT AI agent experiment produce mixed results?
The NYT's July 2026 experiment found that AI agents handled some office tasks well but failed on tasks requiring company-specific knowledge — the right files, internal policies, client history, and team conventions. The models were capable; the missing ingredient was business context.
What is the CMU TheAgentCompany benchmark?
TheAgentCompany is a simulated office environment built by Carnegie Mellon University researchers to test AI agents on real-world tasks. Their findings showed most agents would make poor office workers — getting lost looking for files, inventing information they did not have, and completing tasks incorrectly due to missing internal knowledge.
What is a business context layer for AI?
A business context layer is a system that ingests a company's data from the tools it already uses — Slack, Gmail, Notion, HubSpot, Salesforce, and others — and makes that knowledge available to AI agents at query time. It sits between your company's data and your AI tools, providing permissioned, source-cited, current context without requiring fine-tuning or a custom RAG pipeline.
How does Model Context Protocol (MCP) relate to AI agents doing office work?
MCP is the protocol that lets AI agents query external knowledge sources in a standardized way. When a company's knowledge is exposed as an MCP server, any compatible AI agent — Claude, ChatGPT, Cursor — can query it directly. This is the architectural piece that lets agents answer questions about your specific business rather than defaulting to generic world knowledge.
Is fine-tuning a better solution than a context layer?
No — fine-tuning bakes a static snapshot of knowledge into a model and requires expensive retraining every time your business changes. A context layer stays current, is permissioned per user or team, and cites sources. For company-specific operational knowledge, fine-tuning is the wrong tool for the job.