Every vendor selling AI infrastructure now claims to offer a context layer. As DataHub's team observed, consensus on the problem has not produced consensus on the solution — each vendor defines "context layer" to match whatever they already sell. The result is that teams evaluating options end up comparing things that share a name but not a function.
This post is an evaluation framework, not a survey. Five primitives decide whether an AI context layer actually works when agents are running against real company data. Get any one wrong and the system fails — sometimes loudly, sometimes in ways you only discover after an agent has acted on stale or unauthorised information.
Key takeaways
- An AI context layer has five non-negotiable primitives: permissioning, source citation, freshness, scoping, and provenance. All five must hold simultaneously.
- Permissioning is the highest-stakes primitive. A context layer that exposes data beyond what a user is allowed to see is a liability, not an asset.
- Source citation is what separates a context layer from a black box. Without it, agents cannot be audited and outputs cannot be trusted.
- Freshness is an architectural property, not a feature. A layer that serves stale context produces confident wrong answers.
- Scoping and provenance are the primitives most often skipped — and the ones that cause the most expensive failures.
What an AI context layer actually is
An AI context layer is the governed infrastructure that sits between a company's data sources and the AI agents that need to reason about that data. It ingests knowledge from the tools a business already uses, structures and permissions that knowledge, and delivers it to agents at runtime — with enough structure that the agent can act correctly and an auditor can verify why it did.
That definition is narrow on purpose. A semantic layer interprets structured data. A RAG pipeline retrieves documents by vector similarity. A context layer does something different: it makes company knowledge machine-usable across all of those retrieval patterns, with governance attached. Yali Sassoon at Snowplow makes this distinction clearly: a semantic layer and a context layer are both valuable to agents, but they are distinct.
The five primitives below are what separate a context layer that works in production from one that works in a demo.
Primitive 1: Permissioning — who can see what, enforced at retrieval
Permissioning is the highest-stakes primitive because its failure mode is invisible until something goes wrong. An agent that retrieves HR compensation data to answer a sales question, or surfaces a confidential board memo in a general Slack query, has not malfunctioned in any way the model can detect. The model answers confidently. The damage is done.
A production-grade context layer enforces permissions at the point of retrieval, not at the point of ingestion. The distinction matters. Ingestion-time filtering — deciding what goes into the index — is a coarse instrument. It cannot handle the common case where the same document is visible to some users and not others, or where access changes after indexing.
Retrieval-time enforcement means every query is evaluated against the requesting user's actual permissions before any context is returned. This requires the context layer to maintain a permission model that mirrors the source systems: if a Notion page is restricted to the finance team, the context layer must know that and enforce it, even when a general-purpose agent asks the question.
How it fails: The most common failure is what could be called "permission collapse" — a layer that indexes everything and returns everything, relying on the agent's system prompt to police access. System prompts are not access control. They can be overridden, misread, or simply ignored when the model's context window is under pressure.
How to test it: Ask the layer a question that a low-permission user should not be able to answer. Then ask it as a high-permission user. The answers should differ. If they do not, the layer is not enforcing permissions at retrieval.
Gyld handles this with three permission tiers — private, team, and company-wide — applied at indexing time and enforced at retrieval. What a managed company brain looks like on your own apps covers how that plays out across connected sources.
Primitive 2: Source citation — every answer carries its evidence
Source citation is what separates a context layer from a black box. When an agent answers a question using retrieved context, the answer must carry a reference to the source: which document, which Slack thread, which CRM record, and when it was last updated.
This is not a nicety. It is the mechanism that makes agent outputs auditable. Without source citation, a team cannot verify whether an agent's answer was grounded in current company knowledge or confabulated. They cannot tell whether the retrieved source was authoritative or peripheral. And when an agent acts on wrong information, they cannot trace why.
Gartner's framing, as cited by DataHub, puts provenance as one of the three core components of a context layer, specifically "mechanisms to track data sources to insights, decisions and outcomes, supporting continuous improvement and auditability." Source citation is the user-facing expression of that provenance requirement.
How it fails: Many implementations surface answers without citations, or with citations that point to a document title but not a specific passage or version. That is enough to create the appearance of grounding without the substance of it. An agent that says "according to our pricing policy" without linking to the actual policy document is not auditable.
How to test it: Ask the layer a question with a known answer that exists in a specific document. Check whether the response identifies the source document, the specific section, and the date it was last updated. If any of those are missing, the citation is decorative.
This connects directly to the AI agent accountability question: when an agent acts on retrieved context, the organisation needs to be able to show what that context was and where it came from.
Primitive 3: Freshness — context that reflects the current state of the business
Freshness is an architectural property, not a feature toggle. A context layer that serves stale data produces confident wrong answers — and confident wrong answers are more dangerous than obvious errors, because they pass human review.
DataHub's analysis of context layer components identifies "operational state" as a core requirement: AI agents must operate with the most current information. That means the context layer's update mechanism has to be event-driven or near-real-time, not a nightly batch job.
Consider what goes stale and how fast. A Slack conversation about a customer issue may be resolved within hours. A deal stage in Salesforce changes when a call ends. A pricing document updated on a Tuesday afternoon is wrong context for an agent answering a customer question on Tuesday evening. The acceptable staleness window varies by source and by use case, but the context layer has to make that window explicit and configurable — not assume that weekly reindexing is sufficient.
How it fails: The most common failure is a layer that indexes on a schedule and has no mechanism for detecting that a source document has changed between cycles. An agent asked "what is our current refund policy?" retrieves the version from the last index run, which may be days old. The answer is confident and wrong.
A subtler failure is freshness theatre: a system that shows a "last updated" timestamp on retrieved documents but does not actually re-retrieve the content when the source changes. The timestamp creates the impression of currency without the reality.
How to test it: Update a document in a connected source. Wait ten minutes. Ask the context layer a question whose answer depends on the updated content. If the old answer comes back, the layer is not fresh. Note how long it takes for the new answer to appear — that is your actual freshness window.
Primitive 4: Scoping — indexing what matters, not everything
Scoping is the primitive most teams skip when they are moving fast, and the one that causes the most expensive failures at scale. The instinct is to index everything and let the retrieval layer sort it out. That instinct is wrong.
A context layer that indexes indiscriminately creates three problems. First, retrieval quality degrades as the index grows with noise — irrelevant documents compete with relevant ones, and the agent's retrieved context becomes less precise. Second, permission surface area expands: every additional document is another potential exposure. Third, the organisation loses control of what the agent "knows," which makes auditing and correction harder.
Scoping means the team that owns the context layer makes deliberate decisions about what gets indexed, at what granularity, and with what metadata. It is the governance step that makes the layer trustworthy rather than merely comprehensive.
Atlan's framework for a working context layer identifies "governed" as a non-negotiable property — meaning the context layer enforces policies about what knowledge is available to agents, not just what is technically retrievable. Scoping is how that governance is implemented in practice.
How it fails: The failure mode here is what might be called "context sprawl" — a layer that has grown to include draft documents, deprecated policies, personal notes, and test data alongside authoritative sources. An agent asked about the current sales process retrieves a mix of the live playbook, a draft from six months ago, and a Slack thread where someone proposed a change that was never adopted. The agent synthesises across all of them and produces an answer that is partially wrong in ways that are hard to detect.
How to test it: Ask the layer a question that has a clear authoritative answer in one document and a contradictory answer in an older draft or informal source. Check which one the agent surfaces. If it surfaces both and synthesises across them without indicating which is authoritative, scoping is not working.
The post on how to build a company brain — what to connect first covers the sequencing question in detail: which sources to index first, and why starting narrow produces better results than starting broad.
Primitive 5: Provenance — the full chain from source to answer
Provenance is the deepest primitive and the hardest to implement. It is also the one that determines whether a context layer can support enterprise use cases that carry real accountability.
Source citation (primitive 2) tells an agent and a user where a piece of retrieved context came from. Provenance goes further: it tracks the full chain from original source to retrieved context to agent output to action taken. When an agent sends an email based on retrieved pricing context, provenance means the organisation can reconstruct exactly which version of the pricing document was retrieved, when it was indexed, who last updated it, and what the agent did with it.
Gartner's definition, as cited by DataHub, frames provenance as supporting "continuous improvement and auditability" — both of which require the full chain, not just the last link.
Provenance also enables correction at scale. When a context layer serves wrong information and an agent acts on it, the organisation needs to know how many agents retrieved that information, what they did with it, and whether any of those actions need to be reversed. Without provenance, that investigation is manual and incomplete.
How it fails: The most common failure is that provenance is tracked at the document level but not at the passage level. An agent retrieves a specific paragraph from a 40-page policy document. The citation says "Policy Document v3.2." The actual passage that grounded the answer is not identified. When the policy is updated, there is no way to know which agent outputs were affected by the changed passage.
How to test it: Run an agent query that retrieves context from a known source. Then update that source. Check whether the system can tell you which prior agent outputs were grounded in the now-changed content. If it cannot, provenance is incomplete.
This connects to the broader question of why AI agents give wrong answers about your business — most of the time, the failure traces back to a provenance gap: the agent acted on context whose lineage was not tracked.
How the five primitives interact
Each primitive is a necessary condition. None is sufficient alone.
| Primitive | What it guarantees | How it fails silently |
|---|---|---|
| Permissioning | Right people see right context | Permission collapse at retrieval |
| Source citation | Outputs are auditable | Decorative citations without passage-level specificity |
| Freshness | Context reflects current state | Scheduled indexing with no change detection |
| Scoping | Index contains signal, not noise | Context sprawl from undiscriminating ingestion |
| Provenance | Full chain from source to action | Document-level tracking without passage-level lineage |
A layer with strong permissioning but poor freshness will enforce access correctly on stale data. A layer with good freshness but weak scoping will serve current noise alongside current signal. A layer with source citation but no provenance will answer the "where did this come from?" question but not the "what did we do with it?" question.
The evaluation question is not "does this system have these features?" It is "do all five hold simultaneously, under load, with real company data?"
The audit you can run today
Before committing to any AI context layer — built or bought — run these five checks against a representative sample of your actual data:
- Permissioning check: Query as a restricted user for content that a privileged user can access. Confirm the restricted user gets nothing.
- Citation check: Ask a question with a known answer in a specific document. Confirm the response cites the document, section, and version.
- Freshness check: Update a source document. Measure how long before the updated content appears in responses.
- Scoping check: Ask a question where an authoritative answer and a contradictory draft both exist in the index. Confirm the agent surfaces the authoritative one and flags the conflict.
- Provenance check: Run a query, then update the source. Confirm the system can identify which prior outputs were grounded in the changed content.
If any of these checks fails, you have found the primitive that will cause the most expensive production incident. Fix it before you scale.
For teams evaluating whether to build this infrastructure or use a managed layer, the comparison of Gyld against RAG and other approaches covers the build-vs-buy tradeoffs in detail.
Frequently asked questions
What is an AI context layer?
An AI context layer is the governed infrastructure that sits between a company's data sources and the AI agents that need to reason about that data. It ingests knowledge from tools the business already uses, structures and permissions that knowledge, and delivers it to agents at runtime with enough structure that outputs can be verified and audited.
How is an AI context layer different from RAG?
RAG (retrieval-augmented generation) is a retrieval pattern: find relevant documents by vector similarity, pass them to the model. A context layer is the governed infrastructure that makes retrieval trustworthy — it handles permissioning, freshness, scoping, citation, and provenance across all retrieval patterns, including but not limited to vector search. RAG is one mechanism a context layer can use; it is not the layer itself.
Why does permissioning need to happen at retrieval, not ingestion?
Ingestion-time filtering decides what enters the index. It cannot handle cases where the same document is visible to some users and not others, or where access rights change after indexing. Retrieval-time enforcement evaluates every query against the requesting user's current permissions before returning any context. Only retrieval-time enforcement handles dynamic access correctly.
What does "freshness" mean for a context layer in practice?
Freshness means the context layer detects when a source document changes and updates its index before the next scheduled cycle. For most business use cases, a nightly batch job is too slow — deal stages change during a call, policy documents are updated during the day, Slack threads resolve within hours. A production context layer needs event-driven or near-real-time update mechanisms, not scheduled reindexing.
What is the difference between source citation and provenance?
Source citation answers "where did this piece of context come from?" — it identifies the document, section, and version that grounded an agent's answer. Provenance answers "what happened to every output that was grounded in this source?" — it tracks the full chain from source to retrieved context to agent output to action taken. Citation is the user-facing expression of grounding. Provenance is the audit trail that enables correction at scale.
How does scoping affect retrieval quality?
An index that contains everything — drafts, deprecated policies, personal notes, test data — forces the retrieval layer to compete authoritative content against noise. The agent synthesises across whatever it retrieves, which means a mix of authoritative and peripheral sources produces answers that are partially wrong in ways that are hard to detect. Scoping keeps the index to deliberate, governed content, which improves both retrieval precision and the trustworthiness of agent outputs.
Can I build these five primitives myself, or do I need a managed layer?
All five can be built. The question is whether the engineering cost is justified relative to the business problem. Permissioning and provenance in particular are hard to get right and hard to maintain as source systems change. Teams that have built their own context layers typically find that freshness and provenance require ongoing maintenance as connected apps update their APIs and data models. A managed layer absorbs that maintenance cost; a self-built layer keeps it in-house.
Related reading
- How to Build a Company Brain: What to Connect First — the sequencing decisions that determine whether your context layer starts with signal or noise.
- Why AI Agents Give Wrong Answers About Your Business (And How to Fix It) — most wrong answers trace back to one of the five primitives failing.
- AI Agent Accountability in Business: Who's Liable When It Goes Wrong? — why provenance and source citation are legal and operational requirements, not just engineering preferences.
- How to Ground AI in Your Company Data (So Agents Stop Guessing) — practical steps for connecting real company sources to the agents you already use.
If you want a context layer where all five primitives are handled — permissioning, citation, freshness, scoping, and provenance — without building the infrastructure yourself, start building your company brain with Gyld.