The assumption most people make when they build an AI assistant is that the goal is to answer questions correctly. That framing misses something: the queries that return no good answer are just as informative as the ones that do. KRE, the Knowledge Refinement Engine in KaryoSpace, is the system that treats retrieval failures as structured signals rather than exceptions to silently swallow.
KRE is not a knowledge base. A knowledge base is where you put things you already know you need. KRE is the system that tells you what you're missing. The distinction matters because the failure mode of most enterprise AI assistants is not that they answer questions wrongly. It's that they confidently answer questions about things they have no data on, or they silently return empty results and leave the user without recourse. KRE closes that gap by making retrieval failure a first-class event.
A knowledge base is reactive: you fill it with what you think people will need. KRE is observational: it watches what people actually ask and tells you where your coverage ends.
Every query submitted through KaryoSpace's AI assistant is stored in the user_queries MongoDB collection. The document structure has a sources field (array of source badges that contributed to the answer) and a retrieved_count field. When the RAG pipeline runs and finds no relevant documents, sources is an empty array and retrieved_count is zero. That's the signal. Not an error, not a timeout, not a 500, just an empty sources array. KRE's job is to watch for those documents and classify them.
{
"_id": ObjectId("..."),
"user_id": "linda.chen@samscorp.xyz",
"org_id": "samscorp.xyz",
"query": "what did John say about the AWS migration timeline?",
"response": "I don't have specific information about that discussion...",
"sources": [],
"retrieved_count": 0,
"timestamp": ISODate("2026-06-14T09:23:11Z")
}When sources is empty, the query is a gap candidate. The content that would have answered it does not exist in any indexed collection.
An empty sources array is not a bug. It's a data point. The pipeline worked correctly: it searched, found nothing relevant, and said so. KRE's job is to aggregate those data points into a picture of what the org's knowledge system doesn't cover.
Not all retrieval failures are the same kind of gap. KRE classifies them into four categories by analyzing the noun phrases in the failed query:
| Category | Example Query | Root Cause | Resolution |
|---|---|---|---|
| Missing people | "What did Marcus think about the proposal?" | Marcus is not a KaryoSpace user or his email is not connected | Invite Marcus or connect his email account |
| Missing projects | "What's the status of the Sentinel project?" | Sentinel has no Jira issues, Confluence pages, or email threads in connected sources | Connect the system where Sentinel lives, or create a knowledge document |
| Missing processes | "How do we handle customer escalations?" | The process lives in someone's head, not in any indexed document | Write and index a process document |
| Missing context | "What was decided in this morning's standup?" | Very recent content not yet in the index due to sync interval | Wait for next sync (15 min for email, daily for Confluence) or trigger manual sync |
The classifier uses a simple keyword-extraction approach, not a second LLM call. Adding an LLM classification step for every failed query would double the latency for queries that already failed. Instead, gap classification runs asynchronously after the response is returned to the user.
Async classification means the user never waits for gap analysis. The response goes back immediately. KRE processes the failure in the background, updates the gap index, and surfaces the result in the admin dashboard on the next refresh. Speed for the user, signal for the operator.
KRE surfaces gaps through the AIO (AI Inference Observability) admin dashboard. The /admin/ai-queries page already exists to show all AI interactions. KRE adds a "Gaps" tab that filters to queries where sources is empty, groups them by noun phrase clusters, and shows frequency counts. An admin can see: "12 queries this week mentioned 'Sentinel project' and all returned no sources." That's an actionable signal: either connect the system where Sentinel lives, or create a knowledge base document that describes it.
The gap-closure loop is:
KRE detects a cluster of failed queries around a topic. The cluster threshold is configurable, defaulting to 3 or more queries sharing a noun phrase within a 7-day window.
The admin sees the cluster in the Gaps tab with query examples, frequency counts, and the first and last time the gap appeared. The queries themselves are shown verbatim so the admin understands exactly what users were trying to find.
The admin either (a) connects the missing data source, (b) writes a knowledge base document, or (c) ensures the relevant people's email is connected to the org workspace.
The next time a similar query arrives, the RAG pipeline finds the content and the gap closes. KRE tracks closure automatically by monitoring whether subsequent queries in the cluster start returning non-empty sources.
What makes KRE useful as a long-term system is that it gets better the more it's used. Early in deployment, many queries fail because the org's data isn't fully connected. Each failure surfaces a gap. Each gap that gets closed reduces future failures. The system converges toward complete coverage of the org's actual knowledge domain, driven by real user queries rather than by someone trying to predict in advance what to document.
This is fundamentally different from asking users to fill out a knowledge base. Users ask questions about what they actually need to know. KRE turns that implicit demand into an explicit content roadmap.
At week one, an org might see 40% of queries returning empty sources. At week eight, after following the gap-closure loop consistently, that number typically falls below 10%. The system becomes measurably more useful over time, not because the model improved, but because the knowledge coverage improved. That's a distinction worth preserving: the AI didn't get smarter, the org's indexed knowledge got more complete.
Three improvements I haven't built yet:
I built the detection and dashboard before the notification and ticketing layers intentionally. You need to see the gaps clearly before you automate the response to them. Automating gap-to-ticket before understanding gap patterns would flood the project tracker with low-signal noise.
The insight that drove KRE's design is simple: you can't improve what you can't see. Most AI assistants hide their ignorance behind confident-sounding responses or silent empty states. KRE treats ignorance as data. Every retrieval failure is a message from users about what the org knows that the system doesn't. The longer you run it, the more complete the picture becomes.
KaryoSpace is live at karyospace.com. Questions about knowledge gap detection or enterprise AI systems: sumanakkisetty@gmail.com
Thanks for reaching out. We'll get back to you shortly.