RAG vs OKF: A Product Manager's Guide to Choosing How Your AI Knows Things

The support bot passed every demo. Then, three weeks after launch, a customer asked how many days they had to return a product. The bot said 30. The policy had changed to 15 in April.

Nobody had forgotten to upload the new policy. It was there, in the knowledge base, right next to last year's version. The bot simply retrieved the older document, because its wording happened to sit closer to the customer's question. The answer was fluent, confident and wrong, and nothing in the system flagged it.

Now picture being the PM (Product Manager) on that bot. Support has spent two days walking angry customers back from a promise the company never made. Your head of operations wants to know two things by Friday: how do we make sure this never happens again, and what will it cost?

If you are building anything with AI in it, you will end up in that meeting. For the last three years, the default fix has been RAG (Retrieval-Augmented Generation). In June 2026, Google Cloud published a different answer: the Open Knowledge Format, or OKF.

This guide explains both in plain language, where each one earns its place, and a six-question test you can run in your next planning meeting to decide which one your product needs. (Spoiler: most serious products end up using both, but in a specific order.)

The one-sentence difference

RAG helps an AI find things. OKF helps an AI know things.

RAG is built for the situation "the answer is somewhere in this huge pile of documents, go look." OKF is built for the situation "here is what we already know to be true, written down clearly, go read it."

Hold on to that distinction. Every decision in the rest of this post comes back to it.

It also explains the most common wrong turn. When a RAG bot gets a policy wrong, the instinct is "we need better RAG": a bigger model, smarter chunking, a reranker. Those help at the margins, but they do not fix the underlying issue. The return window is not something the bot should be searching for. It is something the company has decided. The real work is separating the knowledge you are willing to stand behind from the knowledge you are willing to search through.

What is RAG?

RAG is a technique that gives a large language model access to your documents at the moment it answers a question. It works in three steps:

1. Index. Your documents (PDFs, wiki pages, tickets, contracts) are split into small pieces called chunks. Each chunk is converted into a list of numbers, called an embedding, that captures its meaning. These are stored in a vector database.
2. Retrieve. When a user asks a question, the question is converted into an embedding too, and the system pulls the chunks whose meaning is closest to it.
3. Generate. Those chunks are pasted into the model's prompt, and the model writes an answer based on them.

The appeal is obvious. You do not have to retrain a model, you do not have to restructure your documents, and you can point it at millions of pages. That is why RAG became the default architecture for enterprise AI assistants, and why we have covered its many variants in our guide to 26 types of RAG systems.

What is OKF?

OKF is an open specification, published by Google Cloud on 12 June 2026, for handing curated knowledge to AI agents as plain Markdown files. It was written by Sam McVeety and Amir Hormati of Google Cloud, released under the Apache 2.0 licence, and is deliberately not tied to Google's products. You do not need a Google Cloud account, a vector database or any SDK to use it.

An OKF "bundle" is simply a folder of Markdown files:

• One concept per file. A metric, a policy, a table, a runbook, an API, a business rule.
• A small block of labelled fields at the top of each file (called YAML frontmatter). Only one field is required: type. Recommended fields include title, description, resource and tags.
• Normal Markdown links between related concepts, so the knowledge forms a navigable map rather than a pile.
• Two reserved files: index.md, a table of contents the agent reads first, and log.md, a running history of changes.

The second version of the spec (v0.2) added what matters most to a PM: trust metadata. A concept can now record who generated it, who verified it and when, what sources it came from, whether it is draft, stable or deprecated, and a stale_after date after which it should no longer be trusted. A concept verified by a person (marked human:) is treated as human-reviewed; one verified only by software is machine-confirmed; one with no verification is unverified.

Here is what the return-policy concept from our opening scene might look like in OKF:

---
type: policy
title: Customer Return Window
description: Number of days a customer has to return a product.
status: stable
verified:
 - { by: "human:cx-policy-team", at: 2026-04-01 }
stale_after: 2026-10-01
sources:
 - resource: /policies/returns-v4.pdf
 title: Returns Policy v4 (April 2026)
---
# Return window
Customers may return products within **15 days** of delivery.
Supersedes the 30-day window in Returns Policy v3.

There is exactly one file that states the return window. It says who signed off on it, when it expires, and what it replaced. An agent reading it cannot confuse it with last year's version, because last year's version is not a competing chunk. It is a line of history.

Where OKF came from

OKF formalises a pattern that the AI researcher Andrej Karpathy described in April 2026, which he called the "LLM wiki." His idea: instead of making the model rediscover your knowledge from raw documents on every question, have it build and maintain a persistent set of interlinked Markdown pages once, then keep them current. As his gist puts it, "the cross-references are already there."

Many teams had started building their own versions of this. OKF's contribution is a shared format, so a knowledge bundle written by one team or tool can be read by any agent without translation.

RAG vs OKF at a glance

Benefits of RAG

• Scale without curation. You can make ten years of tickets, contracts or research reports searchable without anyone rewriting them.
• Handles the unknown unknowns. Users ask questions you never anticipated. Semantic search can surface a relevant paragraph from a document nobody remembered existed.
• Fast to start. A working prototype can be built in days with off-the-shelf tools.
• Keeps up with volume. New documents flow in through a pipeline. No editor has to approve each one.
• Cites sources naturally. Because answers come from retrieved chunks, you can show users the exact passage the answer was built from.

Where RAG breaks

RAG's weaknesses show up precisely on the questions where being wrong is most expensive: questions about facts your organisation has already decided.

• Similar is not the same as correct. Retrieval rewards the chunk whose wording is closest to the question, not the one that is current or authoritative. That is exactly how our bot chose the old return policy.
• Chunking cuts through meaning. A definition on page 4 and its exception on page 11 can end up in different chunks, and only one gets retrieved.
• Failures are silent. A bad retrieval does not throw an error. It produces a confident paragraph that looks exactly like a good answer.
• Relationships are invisible. "Which dashboards use this metric?" or "What replaced this API?" require connections that a bag of chunks does not store.

Benefits of OKF

• One source of truth per fact. Each concept lives in one file. There is nothing for an older version to compete with.
• Trust is visible. The agent (and your user) can see whether an answer rests on a human-reviewed definition or an unverified draft. That is a product feature, not just an engineering detail.
• Knowledge expires on purpose. stale_after lets a policy or metric declare its own shelf life, so stale knowledge becomes a visible warning rather than a silent error.
• Readable by humans, reviewable like code. Because it is plain Markdown, product, legal and domain experts can read and edit it. Kept in Git, every change has an author, a diff and an approval.
• No lock-in. It is a format, not a platform. The same bundle works across models, agent frameworks and clouds, and can be exposed through MCP (Model Context Protocol) or a simple file-reading tool.
• Cheap to run. No embedding pipeline and no vector database. The cost moves from infrastructure to curation.

Where OKF breaks

• It needs owners. Somebody has to write, verify and retire concepts. If no team owns the knowledge, the bundle rots, just more visibly.
• It does not scale to the long tail. You will not hand-curate a million support tickets. Reading files also consumes the model's context window, so very large bundles get expensive and slow.
• It is new. The spec is at an early version and explicitly described by its authors as a starting point. Expect it to change, and expect tooling to be thinner than RAG's mature ecosystem.
• It only knows what you have written down. Novel questions outside the curated set need another path.

Use cases: where each one fits

Strong fits for RAG

• Searching historical support tickets to find how similar customer problems were solved before.
• Contract and document review across thousands of vendor agreements, looking for unusual clauses.
• Research assistants over reports, papers, or market intelligence that grow every week.
• Enterprise search over email archives, meeting transcripts and shared drives.
• Public-sector file archives, such as finding precedent across years of noted files and circulars.

Strong fits for OKF

• Business metric definitions. What exactly counts as an "active user," "churned customer" or "qualified lead." Analytics agents that get this wrong produce beautiful charts of the wrong number.
• Policies and business rules. Return windows, leave policies, discount approval limits, eligibility criteria for a scheme.
• Data catalogues. What each table contains, how tables join, which column is the trusted one. Google's own sample bundles cover public datasets such as GA4 e-commerce, Stack Overflow and Bitcoin.
• Runbooks and incident playbooks. Steps an on-call engineer or an agent should follow, verified by the team that owns the system.
• API and product documentation, including deprecation notices and what replaced what.
• Regulated facts. Anything where you need to show an auditor who approved the knowledge the AI used, and when.

The PM's decision guide: six questions

Run your knowledge through these questions. Do it per type of knowledge, not per product. A single product usually contains several kinds.

1. How big is it?

If the knowledge fits into hundreds or a few thousand clearly named concepts, OKF is realistic. If it is tens of thousands of documents and growing, it is RAG territory.

2. How often does it change, and who changes it?

Knowledge that changes through a decision (a policy update, a new metric definition) suits OKF, because the decision-maker can update one file. Knowledge that changes through accumulation (new tickets, new contracts) suits RAG.

3. What does a wrong answer cost?

If a wrong answer means a refund dispute, a compliance breach, a bad financial decision or a misinformed citizen, you want OKF's explicit, verified knowledge for that slice. If a wrong answer means the user rephrases and searches again, RAG's occasional misses are acceptable.

4. Is there a clear owner?

OKF only works if a named team owns each area of knowledge and will verify it on a schedule. If you cannot name the owner in the meeting, you have three routes:

• Name one. Often the owner exists informally (the person everyone asks). Make it formal, then proceed with OKF.
• No owner yet, but there should be. Keep that area on RAG for now, and log it as a governance gap to close next quarter. Be honest in your product requirements that answers in this area are unverified.
• The knowledge is genuinely distributed, like customer feedback or field notes. Nobody will ever "decide" it. RAG is the right tool, permanently.

5. Do relationships matter?

If users ask questions like "what depends on this" or "what replaced that," you need explicit links. OKF stores them; RAG has to guess them.

6. Do you need to prove where an answer came from?

If legal, audit or a regulator will ask "which approved source did the AI rely on," OKF's verified and sources fields give you that trail by default. With RAG, you have to build it.

Reading the result

The answer for most products: both, in the right order

OKF does not replace RAG. It replaces RAG for the slice of knowledge that is small, stable and already decided. For everything else, retrieval is still the right tool.

The architecture most teams converge on looks like this:

1. A curated core in OKF. Policies, definitions, rules, schemas: the facts your organisation stands behind.
2. A long tail in RAG. Archives, tickets, transcripts, reports: the material you search, not the material you certify.
3. A router in front. A lightweight step that decides, per question, whether to read from the curated core, search the archive, or both. When both are used, the curated core wins any conflict.

The router sounds like heavy infrastructure, but the first version rarely is. For a small curated core, it can be as simple as always giving the agent the OKF index.md plus a tool to open concept files, alongside the usual search tool, with an instruction that verified concepts override search results. Build a dedicated classifier only when the curated core grows large enough that the index itself becomes expensive to read on every question. Budget engineering time for the conflict rule and its logging; that is where the real design decisions sit.

The order of the build matters. Many teams start with RAG because it is fast, then discover in production that the most painful errors cluster around a surprisingly small set of facts. Those facts are your first OKF concepts. Pull them out of the retrieval pile, give them owners and expiry dates, and let RAG handle the rest.

Three worked scenarios

Scenario 1: The HR policy assistant

A 5,000-person company wants employees to ask an assistant about leave, travel allowances and reimbursement. There are roughly 60 policies, each owned by HR, each updated a few times a year, and every wrong answer becomes a grievance.

Verdict: OKF first. Sixty owned, high-stakes concepts is exactly what the format is for. Add RAG later over past HR tickets, so the assistant can also answer "how did we handle a case like mine?"

Scenario 2: The legal contract explorer

A legal team wants to search 40,000 historical vendor contracts for indemnity clauses that differ from the standard template.

Verdict: RAG. The corpus is huge, unstructured and not being "decided" by anyone. But the standard template itself and the list of approved clause variations belong in OKF, so the system knows what "normal" means before it goes looking for exceptions.

Scenario 3: The analytics agent

A product team wants an agent that answers questions like "how did weekly active users change after the pricing update?" by writing queries against the data warehouse.

Verdict: OKF, clearly. The agent's biggest risk is not missing a document; it is using the wrong definition of "active user" or joining the wrong tables. Metric definitions, table descriptions and join paths are precisely the use case Google designed OKF around.

Scenario 4: You already have RAG in production

This is the most common situation, so it deserves its own plan. Say you run a customer assistant for a lending app, built on RAG over every product document the company has. It mostly works. But when you read the escalated conversations, the painful ones keep circling the same handful of topics: interest rates by product, foreclosure charges, eligibility criteria, grievance timelines.

Verdict: retrofit, do not rebuild. Leave the RAG system running. Write those few topics up as OKF concepts, each verified by the team that owns it (product, credit, compliance) and each with a stale_after date tied to their review cycle. Give the assistant the curated core first, with search as the fallback. Then compare escalations on those topics before and after. If they drop, extend the core to the next cluster. If they do not, your problem was somewhere else, and you found out cheaply.

What to measure

Whichever you choose, define success before you build. Useful metrics for each:

• For RAG: retrieval precision on a fixed test set of real questions, the share of answers with a correct citation, and the rate of answers that users flag or rephrase.
• For OKF: the share of concepts that are human-verified, the number of concepts past their stale_after date, time from a policy change to the concept being updated, and coverage (how many real user questions the curated core can answer).
• For both: a "golden set" of your 50 highest-stakes questions. Build it by pulling the most frequent questions from real logs or support tickets, then adding every question that has ever caused an escalation or complaint. Write the correct answer next to each, signed off by the owning team. Re-run the set before every release and on a fixed monthly date, and have a domain expert review any answer that changed. This one test catches more problems than any dashboard.

Four mistakes to avoid

1. Treating OKF as "RAG is dead." It is not. Anyone telling you a Markdown folder replaces search over a million documents has not operated one.
2. Dumping every document into OKF. An OKF bundle full of unverified, auto-generated pages is just a slower RAG with extra steps. The value is in curation and verification.
3. Skipping ownership. Both approaches fail without owners, but OKF depends on them entirely. Put ownership in the product requirements, not in a hopeful footnote.
4. Letting the archive overrule the core. In a hybrid system, if a retrieved chunk contradicts a verified concept, the verified concept must win, and the conflict should be logged for review.

What to do this week

1. List the top 20 questions your users ask (or will ask) your AI product.
2. Mark each one: is the answer a decided fact or something that must be found?
3. For the decided facts, name the owning team. Write down the ones with no owner.
4. Pick the five highest-stakes decided facts and write them up as OKF-style Markdown files with a type, a verifier and a stale_after date. This takes an afternoon and will tell you more about your product's knowledge problem than a month of vendor demos.

Remember the head of operations asking how to stop the wrong-answer problem, and what it will cost? The PM who handles that meeting well does not arrive with an architecture diagram. They arrive with a list: here are the facts our product stands behind, here is who owns each one, here is when each one expires, and everything else we search for. That list is your answer, and after the exercise above, you will have the first five lines of it.

Run the exercise on your own product this week. It takes about ninety minutes. If you want to build this kind of judgment with practitioners and hands-on projects, knowledge architecture and AI evaluation are core modules in our Applied AI Product Management programme.

Sources: Google Cloud: How the Open Knowledge Format can improve data sharing; OKF specification on GitHub; Andrej Karpathy's LLM wiki gist.