04 / RESOURCES / PLAYBOOK
How to build an AI knowledge base for your business
Give AI your own policies, processes, and product details so it answers from your documents, not guesses. A step-by-step playbook, from choosing the questions to keeping it accurate.
An AI model knows a lot about the world and nothing about your business. Ask it your refund policy, your onboarding steps, or which plan includes what, and it will either say it does not know or, worse, guess.
An AI knowledge base fixes that. It is a curated set of your own documents that an AI assistant reads from before it answers, so it responds with your information and can point to where it came from.
How it works
- Your documentsPolicies, guides, FAQs
- Organized knowledgeClean, current, labeled
- Question askedBy the team or a customer
- Relevant parts foundOnly what is needed
- Answer with sources
When the collection is small, an assistant can read all of it with every question. When it is large, the system searches it first and passes only the most relevant passages to the model. That search-then-answer pattern is called retrieval-augmented generation, or RAG.
Retrieval in one picture
- 01Split
Documents are divided into short passages.
- 02Index
Each passage is stored so it can be found by meaning, not just keywords.
- 03Search
A question pulls back the few passages that best match it.
- 04Answer
The model writes its answer from those passages and cites them.
Step 1: Decide what questions it should answer
Start narrow. A knowledge base that answers one group's top questions well beats one that answers everything badly. Good first candidates:
- Internal how-to: "How do we onboard a new client?" "Where is the template for X?"
- Policy lookups: "What is our refund policy for annual plans?"
- Product and service details: "Which package includes monthly reporting?"
Write down 20 real questions people asked this month. They become your test set in step 5.
Step 2: Gather and clean the source documents
The quality of answers can never be better than the documents behind them.
Step 3: Choose an approach
| AI project or workspace | Built into your tools | Custom setup | |
|---|---|---|---|
| What it is | Upload documents to an AI project (for example, a Claude Project) | AI features in your help desk, wiki, or CRM | Your own search and model, built by a developer |
| Setup effort | Minutes | Low | Weeks |
| Best for | Internal teams | Teams already on that tool | Customer-facing or large collections |
| Control | Medium | Low | High |
The quickest start: create a project in your AI assistant, add instructions, and upload your cleaned documents. In Claude, a Project holds shared instructions and files that every conversation in it can use, and team plans let colleagues use the same project.
Step 4: Structure the content so AI can use it
AI reads documents much like a new employee would. Make them easy to scan:
- One topic per document, with a clear title ("Refund policy: annual plans").
- Headings and short sections instead of long unbroken pages.
- Dates and owners at the top, so it is clear what is current.
- Plain text where possible. Text inside images or complex tables is harder to use.
Then write short instructions for the assistant, for example:
You answer questions for the Acme operations team using only the
documents in this project. If the documents do not contain the answer,
say so and suggest who to ask. Always name the document you used.
Never state prices or make commitments to customers.Step 5: Test with real questions
Ask your 20 questions from step 1 and score each answer:
| Result | What it means | Fix |
|---|---|---|
| Right, with source | Working as intended | Nothing |
| Right, no source | Probably guessed correctly | Strengthen the instructions |
| "I don't know" | Answer is missing | Add or improve a document |
| Wrong | Conflicting or unclear documents | Find and fix the source |
Aim for every answer to be either right with a source, or an honest "I don't know."
Step 6: Set rules for what it may and may not answer
Decide in advance:
- Who can use it (one team, the whole company, customers)?
- What it must never do (quote prices, give legal or medical advice, promise timelines)?
- When it should hand off to a person?
For anything customer-facing, start with the assistant drafting answers for a person to review. See the autonomy ladder in AI in small business operations.
Step 7: Keep it up to date
A knowledge base decays the moment a policy changes and the document does not.
- Give it a named owner.
- Add "update the knowledge base" to the checklist for any policy or process change.
- Review monthly: re-run a few test questions and check what people asked that it could not answer.
Common questions
Is my data used to train the model?
On business and API plans from the major providers, generally not by default, but check the terms of the plan you use before uploading anything sensitive.
How many documents can it handle?
Small collections fit directly into an assistant's context window. Larger ones rely on retrieval, which can handle thousands of documents if they are well organized.
Can customers use it on our website?
Yes, but that is a custom or help-desk build with stricter rules, testing, and monitoring. Prove it internally first.