Customer support
Answers support questions only from the company’s own documentation, cites the source, and says it does not know rather than guessing.
Build effort, not elapsed calendar time. Engagements ran alongside study and employment, so this figure is the work itself rather than the span it sat in.
Interface built in code for this case study — not a client screen capture.
01Business problem
Support teams answer the same questions repeatedly from knowledge that lives in scattered documents, and the chatbots meant to help make things worse by answering confidently and wrongly.
Who feels it
The knowledge a support agent needs is spread across a help centre, an internal wiki, a policy document nobody has opened in a year, and the memory of the longest-serving agent. New agents ask in a group chat. Answers vary by who replies, which is how two customers get two different refund policies.
The obvious fix — put a chatbot on it — is where most implementations fail. A general model with a system prompt will answer questions about a specific business’s returns policy by producing something plausible. Plausible and wrong is the worst possible output in support: it creates a commitment the business did not make, and the customer has it in writing.
The second failure is invisibility. When a bot cannot answer, nobody finds out what it could not answer. The questions that would tell the team what documentation to write are the ones being silently mishandled.
02Product overview
A retrieval-grounded support system that answers strictly from indexed company content with an inline citation for every claim, abstains when the sources do not cover the question, and turns those abstentions into a ranked list of documentation gaps.
Engineering notes
Content is indexed with structure preserved: a chunk carries its document, section heading and audience scope, so an internal-only policy can be retrieved for an agent and never for a customer. Retrieval is scoped by audience before ranking, which makes the permission boundary a property of the query rather than a filter applied afterwards.
Answering is constrained. The generation step receives only retrieved passages and is required to attribute each claim to a passage id; an answer that cannot be attributed is not shown. When retrieval returns nothing above the relevance floor, the system abstains explicitly and hands off, rather than answering from the model’s general knowledge. Abstention is treated as correct behaviour, not a failure to be tuned away.
Every abstention and every low-confidence answer is logged with the question. Those questions are clustered so a support lead sees "fourteen people asked about invoice re-issue and we have no page for it" instead of a stream of individual misses. That turned the system into a content roadmap as much as an answering tool.
03Key features
10 capabilities, of which 4 are model-backed. The rest are ordinary deterministic software, and the distinction is marked so the AI claim stays honest.
Help centre, internal wiki and policy documents indexed with document, section and audience scope preserved on every chunk.
Answers are generated only from retrieved passages, with each claim attributed to a passage that a reader can open.
Every answer shows its sources as links to the exact section, so a customer or agent can verify rather than trust.
When retrieval finds nothing relevant enough, the system says so and offers a handoff instead of producing a plausible answer.
Inside a ticket, agents get a drafted reply with its sources attached, editable before sending.
Internal-only content is retrievable for agents and structurally unreachable for customer-facing sessions.
Unanswered questions are grouped by similarity and ranked by volume, producing a documentation backlog from real demand.
Sentiment, repeat contact and explicit request route a conversation to a human with the full context carried over.
Thumbs and free-text feedback are stored against the answer, its sources and the content version that produced it.
Resolved tickets with verified answers form a regression set, so a prompt or index change can be tested rather than guessed at.
04User flow
The path from the trigger to the finished record, with each step attributed to whoever performs it — a person or the system.
05Architecture
The processing path first, then the layers it runs on, then the constraints that shaped both.
Path through the system
Documents chunked with heading path and audience scope retained.
Embeddings plus keyword index; content version recorded per chunk.
Audience filter applied first, then hybrid ranking with a relevance floor.
Answer generated only from passages, with per-claim attribution required.
Answer, abstain, or escalate based on coverage and confidence.
Abstentions clustered into content gaps; verified answers form the eval set.
Layers
Why it is shaped this way
06Technical decisions
Every decision here was contested by a reasonable alternative. The trade-off column is the part usually left out.
Why
Instructing a model to cite is not a guarantee. Validating that every cited passage id exists in the retrieved set — and rejecting the answer otherwise — converts a request into an enforced property.
Trade-off
Some correct answers are rejected because attribution was malformed, so the abstention rate is higher than it strictly needs to be. In support, that is the right direction to err.
Why
Support questions contain exact tokens that embeddings blur — order numbers, plan names, error codes, product SKUs. Keyword matching handles those precisely while vectors handle paraphrase.
Trade-off
Two indexes to maintain and a fusion step to tune, in exchange for not failing on the most specific and most common queries.
Why
A citation has to point somewhere a person can read, and stale citations destroy trust faster than a wrong answer. The version stamp means a re-indexed document invalidates its old citations rather than silently redirecting them.
Trade-off
Re-indexing has to reconcile versions instead of replacing the index wholesale.
Why
The dangerous failure is not a missing answer, it is a fluent answer assembled from weakly related passages. A floor makes the boundary explicit and tunable.
Trade-off
The floor needs periodic calibration against the evaluation set, since it interacts with both content coverage and retrieval quality.
07Challenges
Not the setup work. These are the problems where the first implementation was wrong and had to be reconsidered.
Approach
Audience scope is a required field on every chunk and a required parameter on every retrieval call, applied as a pre-filter in the query. The customer-facing session type cannot construct a retrieval call with internal scope, so the restriction is enforced by types rather than by discipline.
Outcome
The permission boundary sits in the query path. Internal passages never enter the model context for a customer session.
Approach
Attribution is per claim, not per answer, and the validator checks that each attributed passage was in the retrieved set. Coverage is assessed across the claims rather than the answer as a whole, so a partially supported answer abstains on the unsupported part.
Outcome
Partial knowledge produces a partial answer plus a handoff, instead of a complete-sounding answer with an invented middle.
Approach
Unanswered questions are embedded and clustered, and each cluster is summarized into a single canonical question with its volume and example phrasings. The dashboard ranks clusters by volume and recency.
Outcome
Documentation work became demand-driven. The gap list is the backlog, in order.
08Business value
Stated qualitatively on purpose. Invented percentages are the easiest thing to put on a portfolio and the easiest thing to see through.
Operational effect
What would change at scale
09Interface
Dense operational views rather than dashboards. These are used for hours at a time, so the priorities are legibility, keyboard flow and state that is never ambiguous.
Help widget answer
A cited answer with source links to the exact section, and an explicit handoff when coverage is insufficient.
Agent ticket view
Conversation, retrieved passages with relevance scores, and a drafted reply the agent edits before sending.
Content gaps
Clustered unanswered questions ranked by volume, each with example phrasings and a link to the affected area.
Answer audit
A past answer with the sources and content version that produced it, alongside the feedback it received.
The screen above is built in HTML and CSS for this case study. It reproduces the layout, states and vocabulary of the real build without exposing client data, which is why it exists rather than a screenshot. It is evidence of design and of the decisions behind it. It is not a photograph of a deployed system, and no part of it is a capture of anyone else’s product.
10Technologies
Chosen for the shape of the problem, not for novelty. Anything unusual is justified in the decisions section above.
11Technical preview
There is no repository link on this site. What is available instead is the module structure, the data model, the interface contracts and annotated excerpts written for this case study — enough for a technical reviewer to judge the engineering.
Technical preview
Selected implementation details are available for technical review: component structure, data models, architecture notes and sanitized code excerpts. Some client-specific source material stays private because of confidentiality. Enter the project access keyword if you have been given one — the same one works across every case study.
A presentation convenience, not authentication. This is a static site, so the keyword ships to your browser with the page and anyone reading the bundle can find it — worth saying plainly rather than dressing it up. Nothing confidential is stored behind it.
Inside this section
For private client work, contact me for a walkthrough.
Next step
Send the process, the constraints and the deadline. You will get an honest scope, an architecture sketch and a timeline before any commitment.