Why onboarding docs fail
They are not missing. They are written by the person who built the thing, for someone who already understands it, and filed under a title only the author would search for. A new joiner does not know that 'egress posture' is the page about why their curl times out. The gap is vocabulary, not content, and no amount of reorganising the wiki closes it.
Chunking is the whole game
Naive fixed-size chunking splits a runbook mid-procedure, so retrieval returns step 4 of 7 with no indication that steps 1-3 exist. Chunking on document structure — headings, procedure boundaries, code blocks kept whole — and carrying the parent heading into every chunk's metadata is what turns plausible answers into correct ones.
# keep the heading trail with each chunk
chunk.metadata = {
"source": path,
"breadcrumb": " > ".join(heading_stack),
"updated": mtime,
}The breadcrumb goes into the prompt and into the citation. It's how the model knows a chunk is a step inside a larger procedure, and how the reader knows where the answer came from.
Always cite, and show the age
An onboarding bot that answers confidently from a doc last touched in 2021 is worse than no bot — it teaches the new joiner something false and they carry it for months. Every answer carries the source link and the last-modified date, and anything older than a threshold is flagged in the response rather than silently trusted.
Refusing is a feature
Tuned to say 'that isn't in the docs' rather than reason its way to something plausible. For onboarding specifically, a confident wrong answer costs more than a shrug, because the reader has no context to catch it.
# reject weak retrievals before they reach the model
hits = store.search(q, limit=8)
if not hits or hits[0].score < THRESHOLD:
return "Not in the docs I have. Try #platform-help."A similarity floor is cruder than it sounds and catches most hallucination-by-thin-context, which is the dominant failure mode for internal RAG.
Keeping it current
A stale index is the failure everyone hits in month two. Re-indexing runs on a schedule and on document-change webhooks, so a merged runbook edit is searchable within minutes rather than at the next manual rebuild.