The vector index we built, measured, and switched off
A governed generative-AI platform for a regulated financial institution: four-eyes content approval, division-scoped retrieval, and a retrieval stack that measurement reshaped.
Executive summary
A regulated financial institution needed a generative-AI assistant its risk function could sign off on. The hard part was never the model. It was proving who may ask what, which documents an answer is allowed to draw on, and who approved that document being visible in the first place. We built a three-surface platform (an ingestion and retrieval backend, an employee assistant, and a governance console) in which the institution's existing maker-checker control model is the architecture rather than a layer on top of it. Along the way an offline retrieval study overturned the design: a hybrid vector and keyword stack lost to plain keyword search over model-generated page summaries, and we shipped the simpler thing.
Business context
The institution operates under supervisory expectations that treat an unexplainable automated answer as an unacceptable one. Its knowledge lives in long, densely formatted documents: policies, procedures, product terms and disclosures, full of tables, charts and multi-page sections where only the first page carries a heading. Those documents belong to different divisions, and one division's material is not automatically readable by the rest of the organisation. Any assistant had to respect that boundary inside retrieval, not merely in the interface, and every document reaching the assistant had to have been approved by someone other than the person who uploaded it. Staff also work bilingually, so the same question arrives in two languages and has to reach the same evidence.
The challenge
Staff could not find what the institution already knew. The answers existed, filed across long policy and product documents owned by separate divisions, but finding them meant knowing which document to open and reading past the tables and charts to get there. The obvious fix, an assistant over the whole corpus, was the one thing the risk function would not approve: it could not be shown who was allowed to see which document, who had approved that document, or where a given sentence in an answer had come from. An assistant that could not answer those three questions was not deployable, however good its answers were.