A water utility's institutional knowledge sits in documents: design reports, as-built drawings, operating manuals, incident logs, consent conditions. The people who wrote them have often retired. Making that corpus answerable is one of the clearest AI cases in the sector.
There are two architectures on offer, and they are not equally suited to it.
Retrieval augmented generation
RAG keeps the documents where they are, indexes them, retrieves the passages relevant to a question, and asks the model to answer using those passages. The model is not modified.
For utility documents this has four properties that matter:
- Citations. The answer points at the page it came from. An engineer can check it. For anything feeding a design decision or a regulatory position, an uncitable answer is not usable.
- Currency. A revised manual is reindexed in minutes. Knowledge changes constantly in operations.
- Access control. Retrieval can respect who may see which document. Weights cannot.
- Refusal. If nothing relevant is retrieved, the system can say so instead of inventing.
Fine-tuning
Fine-tuning adjusts model weights on your data. It is the right tool for teaching a model a form rather than a fact: a house report structure, a consistent tone, a domain vocabulary, a rigid output schema.
It is the wrong tool for facts. Fine-tuned knowledge cannot be cited, goes stale the moment a document is revised, cannot be access controlled, and fails silently: the model produces a confident answer in the right style with the wrong number. In a sector where a wrong abstraction limit or a wrong pipe diameter carries real consequences, that failure mode is disqualifying on its own.
The usual answer
RAG for the knowledge, fine-tuning only if output format is a persistent problem after the retrieval works. In most utility projects the second step is never needed, and teams that start with fine-tuning generally end up rebuilding as RAG once someone asks where an answer came from.
The harder problem is not the architecture. It is that half the corpus is scanned PDFs of drawings, and retrieval quality depends entirely on how well that content is extracted. Budget for the document processing, not for the model.
The same discipline applies here as to a telemetry store: the value is in the data layer being right, and the clever part on top is comparatively cheap.