The scattering is real, and nobody planned it
Documentation rarely ends up in one place on purpose. A support team builds a public help center for customers. Engineering keeps its own notes in a wiki. A ticketing tool accumulates internal macros and resolution notes that never get copied anywhere else. Somewhere there's a Slack channel where the actual answer to a recurring question was worked out once, months ago, and never written down again. Each piece made sense on its own. Together, they mean "the documentation" is really four or five different documentations that don't know about each other.
That's worth naming before you add a chatbot to any one of these, because a chatbot doesn't inherit your mental map of where things live. It only knows what it was actually built from.
A chatbot searches its corpus, not "your documentation"
"We trained it on our documentation" sounds like it covers everything you'd call documentation. In practice, someone pointed it at one thing, usually whichever source was easiest to connect: the public help center, because it's a website you can crawl, or an export from one wiki, because that's where the setup happened to start. The chatbot then searches exactly that, and nothing outside it. It isn't withholding the rest. It was never shown the rest.
You: How do we handle a refund for a customer who's on an annual plan but wants to cancel after month 3?
Chatbot: I don't have anything on prorated refunds for annual plans after month 3. Our billing team can confirm the exact policy for you.
That answer might be completely honest and still be missing the point. The real policy could be sitting in a ticketing macro that support agents use every day, just not in the help center this chatbot was built from. The chatbot didn't get anything wrong. It searched the one place it had, and told you plainly when that place didn't cover it. The gap is in what it was connected to, not in how it answered.
Two failure modes, and only one of them is honest
When a chatbot only sees part of your documentation, there are two different ways that can go, and they're not equally bad.
The honest version: it answers from what it has, and says so plainly when a question falls outside that. You saw this above. It's a real limitation, but it's a visible one. You know to go check somewhere else.
The quieter, worse version: it answers anyway, using whatever's closest in the one source it has, even when the actual answer lives somewhere else entirely. A chatbot built only from a general help center might answer a question that has a more specific, more current answer sitting in a ticketing macro, and you'd have no way to tell the difference from the outside. It isn't lying. It's confidently applying the wrong scope to a question that needed a different one.
The difference between those two isn't how much documentation the chatbot has access to. It's whether it treats "outside my corpus" as a reason to say so, or as a gap to paper over with whatever's closest.
What to actually ask before you connect one
A few concrete questions worth asking before you add a chatbot to your site, since "it searches your documentation" is a claim you can pin down instead of taking on faith:
Which specific sources is it built from? Not "your documentation" in the abstract. A URL, a list of exported pages, an integration with a named tool. If the honest answer is "the help center, because that's what we could crawl," that's useful to know before you assume it also covers your internal wiki or your ticketing macros.
What happens when the answer lives somewhere it wasn't connected to? Ask a question you know is only answered in a system the chatbot wasn't built from, and see what happens. A plain "I don't have that, here's who can help" is a chatbot being honest about its scope. A confident-sounding answer that doesn't actually match your real policy is the failure mode from the section above.
Who decided what counts as "the docs"? Usually whoever set up the chatbot picked the easiest source to connect, not necessarily the most complete one. That's a reasonable starting point, but it's worth knowing it was a starting point and not a deliberate decision about where your real answers live.
How we handle this
Our agent's corpus is built from the sources you actually point us at. We don't quietly widen it to whatever else we can find, and we don't claim coverage of a system we were never given. If your help content is split across more than one place, tell us where all of it lives. We can build from a list of pages you give us, as long as we can actually reach them, so a public help center page and a genuinely public wiki page can end up in the same corpus. Anything behind a login has to be exported and handed to us rather than connected to directly, since we don't have a way to sign in on your behalf.
What we don't do is automatically discover and merge documentation from tools we weren't told about. A ticketing system's internal macros or a Slack channel's tribal knowledge won't show up in the corpus unless that content gets pulled out and given to us directly. When a question falls outside whatever the corpus actually covers, the agent is built to say so and point toward a real person, rather than reach into the nearest similar-sounding page and answer with something that only sounds like it fits.
One exception: the chat box on our homepage runs on this site's own pages, a two-page corpus rather than a real help center, so it doesn't link citations there, since a two-page corpus would mostly just point you back to pages you can already see.
The honest scope is the useful one
Scattered documentation isn't a problem a chatbot can quietly solve just by existing. The scattering is a real, separate thing to deal with, ideally before you pick which sources actually feed the chatbot, not something the chatbot fixes on its own by sounding confident. The more useful question isn't "does it know our documentation." It's "which documentation, specifically, and what does it do the moment a question steps outside that."
Related reading
- See all 20 guides in the Actually Helpful library, grouped by what you are actually trying to figure out.
- What "grounded in your docs" actually means, on testing whether an answer traces back to a real source at all, a different question from which sources it had access to in the first place.
- Why "I don't know" is a better answer than a guess, on why admitting a scope gap plainly is the right move even though it can feel like the chatbot is underperforming.
- Why AI chatbots give confidently wrong answers, on why a wrong answer from outside a chatbot's real scope can sound just as sure as a right one.
Ask us what it's actually built from
Before you connect a chatbot to your site, ask exactly which sources feed it. We'll tell you plainly, and you can watch what happens when you ask something that falls outside that.