What Building My Own AI System Taught Me About Governance

Four lessons from a personal project, for assessing AI strategies at oversight level.

As part of my guide for boards on AI governance, cybersecurity, data protection, and data management, I am working on my own local knowledge system. My technical background lets me build and develop it myself. My motivation was pragmatic: I wanted to understand and experience first-hand which practical problems arise in a project like this, so that I can assess them better at board and executive level. I bring experience from digital transformation and technical projects, but I had not been active at the code level for some time. Building it myself is also a way of reactivating that knowledge.

The AI system is meant to deliver the following functions:

• Automatically import scientific sources and roughly categorise them (peer-reviewed yes/no, management summary, key statements).
• Index and manage my entire knowledge base of specialist books, reports, and articles.
• Provide laws, standards, norms, and regulatory notices for quick access and cross-referencing.
• Make relevant information from known authors and publishers specifically accessible, to make research more efficient.

These local data sources are made usable for the AI via a RAG architecture (retrieval-augmented generation). A later extension into an AI board system is planned, and partly already under way: developing governance frameworks, running compliance checks, evaluating financial data, and handling client-specific questions on local infrastructure without cloud exposure.

What this build process revealed does not replace theory, but it complements it operationally and substantively. I am taking four lessons away for assessing AI strategies at oversight level.

When the original vision hits its limits: empirical testing is indispensable

My original plan was a fully local architecture, motivated by data sovereignty. Empirical tests quickly showed that the local system was 17 to 20 times slower than cloud systems on synthesis tasks and delivered epistemically weaker answers, even though the hardware used, based on an NVIDIA GB10 Superchip with 128 GB of memory, is genuinely capable. More powerful hardware could improve performance, but a substantial difference would still require considerably higher investment. Overnight batch processing as a workaround for the performance problems was ruled out, because the use case is interactive. More surprising than the latency was the spread of results across different models, even for simple questions. The original vision hit its limits, and the architecture was rebuilt and adapted to the real situation.

This kind of adaptation works in a small, manageable environment. In larger projects, it is considerably harder, which underlines once more the importance of realistic pilots.

A second aspect came on top, one I knew from theory and could anticipate from other projects, but which carries even more weight in AI projects specifically: the volume, structure, and type of data substantially affect the behaviour of AI systems and the underlying models. Legal texts and texts in different languages produced markedly different results depending on the model and its training, both in content and in quality. Model updates and drift triggered by upgrades changed results significantly between two runs. Both are known in principle. Still, anyone who observes this themselves, and finds that expectation and result no longer match, is surprised nonetheless, because the discrepancies can be significant.

Lesson: strategic AI decisions made without realistic empirical validation are not governance. They are faith in vendor slide decks. In many AI projects I see, realistic pilot data at comparable data volume is missing. Instead there are market studies, polished presentations, theoretical analyses, and roadmaps. That falls well short, because an AI project is not a sprint. It is a marathon. At oversight level, I would require robust pilot results under realistic data conditions for every project or sub-project before giving the green light for production development. If pilot measurements were not completed under realistic conditions, the maturity required for a development release has not been reached.

Small details decide success

One of the phenomena in my system was a single configuration switch that was still active at the command-line level, but that a code change had silently stopped being evaluated correctly anywhere in the pipeline. That this happened to me is partly down to the knowledge-reactivation mentioned at the start. Ultimately, such an error can happen to anyone. In practice, the system re-indexed all content on every run. After three runs, the data base had bloated by a factor of three. The error simply slipped through code review. Only empirical tests exposed it: answer quality against a test question set visibly declined. Other phenomena, such as documents with identical file names, had effects on data quality that I had not sufficiently anticipated. These are solvable with filters, additional queries, and indexing logic, but they are errors that cost time and quality, and that, in production, can cost money and possibly create a reputational risk.

Two lessons follow from this:

• Anyone who takes a vendor’s claim that AI agents and systems will be “ready to deploy in a few weeks” at face value does not have a realistic view of the depth of detail such projects require. Even a single use case took several iterations. A company responsible for several production agents needs a multiple of that empirical effort. That applies to data processing within the pipeline just as much as to the security of the data and processes. If a vendor or a project cannot make robust statements on these aspects, caution is warranted.

• A toggle showing as active in a dashboard, or an option set at invocation, is not evidence that the function actually works. In compliance dashboards, in AI safety filters, in audit modes. Wherever a display suggests safety, it is worth asking when that function was last verified through a differential test. The human in the loop needs to be able to rely on the system, its reports, and its dashboards being accurate.

This level of detail belongs in every project plan. An AI strategy that does not name its detail-level risks is closer to a marketing slide than to a decision basis.

Testing is an ongoing task, not a project phase

Data and systems change in production through various influences. Data bases grow, change, lose integrity, or erode; models get updated; interfaces change. Systematic, regular testing must therefore become part of production maintenance. In rule-based systems, predictable outcomes were the norm. With AI systems, testing becomes more demanding and more important than before. Different processing layers can help avoid or at least reduce bias effects. But without recurring tests and verification of results, no robust statement about quality is possible. What remains is only a guess.

For this use case: hybrid is the professional answer

The local system is suited to content subject to data protection requirements and to structured tasks. The cloud system is suited to interactive knowledge work and synthesis. Neither, taken alone, is sufficient for the range of production use cases. The architecture running in production today deliberately uses both, depending on the application context. Confidential client content stays exclusively local, run in batch mode where performance requires it. General research goes to the cloud, wherever that is unproblematic from a data protection standpoint.

A blanket, “simple” one-size-fits-all solution meant to cover every use case is rarely realistic at the start of a project. Anyone who accepts such a claim from a vendor buys either performance problems or data protection risks, and, with a certain probability, disappointed expectations in the end. A workable architecture requires time for careful analysis and empirical testing. In many cases it will be hybrid, with a clear classification rule for which data may be accessed through which system by which users. That rule is not purely technical. It is a governance decision. Anyone who fails to make it has delegated their oversight function to the vendor. This rule, too, must be reviewed systematically.

What I can pass on from this project

I am taking away four observations that transfer to any AI strategy at oversight level.

• Without pilot data from your own organisation, at comparable data volume and under realistic conditions, an AI strategy is not ready for a decision. Presentations and vendor promises are no substitute for your own measurements.

• Depth of detail belongs in the planning, not just in the final report. A single overlooked configuration error can render a system unusable for strategic purposes, without necessarily being obvious right away.

• For more complex use cases, and a serious literature review already counts as one, purely local models are only realistic to a limited extent. Either it requires substantial investment in your own infrastructure, or performance problems loom. Cloud-only, in turn, creates substantial data protection risks. You need both, coordinated with each other.

• Testing and verification are the underrated core discipline. Even data changes alone, without any further adjustments, can change system behaviour. Without recurring tests and verification, there is no statement about quality. There is only a guess. Much of this can be automated. For my own project, for instance, I built an automated test system using several models as evaluators, working through a large catalogue of questions. But human review at the end remains irreplaceable.

AI governance at oversight level is a duty. But it requires the oversight body to engage with the technology at a basic level and build up technology competence, so that it can place factors like the ones above in context. Not as a developer, but as a responsible overseer.

Technology can be delegated. Oversight, and the accountability that comes with it, cannot.