Chatbot Knowledge Base Best Practices — Why Your AI Bot Gives Bad Answers

Short answer: most bad chatbot answers are retrieval failures, not model failures. The bot found the wrong passage, a diluted passage, or nothing. Fix the content: remove boilerplate from the index, write one clear page per question, add the policies that are not on your public site, and inspect what was actually indexed. Then review unanswered questions every week.

Table of Contents

1. How a Knowledge-Based Bot Works
2. Why Answers Go Wrong
3. Preparing the Content
4. The JavaScript Problem
5. Auditing and Maintaining
6. Key Takeaways

How a Knowledge-Based Bot Works

1. Your content is split into chunks and indexed.
2. A visitor asks a question.
3. The system retrieves the chunks that best match.
4. The model writes an answer from those chunks.

If step three returns the wrong chunks, step four cannot succeed. A good model given irrelevant text either says it does not know, or, worse, answers confidently from what it was given.

Why Answers Go Wrong

Preparing the Content

1. One topic per page or section, with a heading phrased the way a customer asks. "How long does delivery take?"
2. Answers in the first sentence.
3. Self-contained sections. Each chunk may be retrieved alone (AI-friendly docs).
4. Customer vocabulary. If customers say "refund" and your page says "return credit", include both.
5. Policies written down. Refund rules, delivery areas, pricing exceptions. These are usually what visitors ask and often not on the public site.
6. Facts as text. Not only in images or PDFs of scanned documents.
7. One source of truth. Remove or redirect duplicates.
8. Dates on anything that changes.

Then write the instructions that set limits. DeepSync's chat docs put it well: be specific about limits, not personality. "Never quote prices — direct pricing questions to the pricing page" is worth more than "be friendly", because it prevents the failures that cost you (AI replies and persona).

The JavaScript Problem

Many chatbot tools build their knowledge base by crawling your site. A crawler that does not execute JavaScript sees an almost empty page on a single-page application. The bot ends up knowing your footer and little else.

There are three ways to get content in. DeepSync's chat widget (Enterprise plan) uses all three (knowledge base):

The snapshot route exists because a session replay tool has already captured the rendered page as a visitor saw it. Whatever tool you use, check whether its crawler renders JavaScript before trusting what it indexed.

Auditing and Maintaining

Inspect the index. Look at the actual chunks stored for a page. This is the reliable way to find navigation menus, cookie banners, and footers that were indexed and now dilute answers, or a page that should not be there at all.

Test with real questions. Take thirty questions from support tickets and site search. Ask the bot. Mark each answer correct, incomplete, wrong, or refused.

Review weekly:

The fourth row is easy to miss. If fifty visitors ask the bot about delivery to their pincode, the product page should say it. The bot's log is a list of what your site fails to communicate (site search gives the same signal).

Keep sensitive content out. Internal notes, other customers' details, and unreleased plans do not belong in a public bot's index.

Re-index after changes. A price change on the site with a stale index produces confident wrong answers.

Key Takeaways

• Bad answers are mostly retrieval problems.
• Remove boilerplate; write one clear section per question.
• Check that JavaScript-rendered content was actually indexed.
• Unanswered questions are a to-do list for your content and your site.

Conclusion

A support bot does not know your business. It knows the text you gave it. Look at that text, fix it, and the answers follow.

Talk to us about DeepSync chat.