When the Board Must Oversee What It Cannot Inspect

Published: 6 May 2026

The oversight paradox in AI systems.

The board’s legal duty of oversight is clearly stated. Art. 716a para. 1 no. 5 OR requires it as a non-transferable duty. Art. 14 of the EU AI Act requires human oversight for high-risk systems. Floridi refers here to the principle of meta-autonomy, the ethical precondition that a human must always be able to override the system.

All three perspectives assume that the board understands the system well enough to recognise risks and intervene. For neural networks and learning systems, that precondition can only be met to a limited extent. This creates a tension that is rarely named in today’s governance practice, and resolved systematically even more rarely.

Two terms are central to what follows:

Inspectability means insight into a system’s decision logic and chain of reasoning.
Demonstrability means the documented effectiveness of the controls, test results, and decision processes the company has built around that system.

Where inspectability is absent, demonstrability must take over its function in oversight.

What board oversight already means today

It would be an oversimplification to claim that classic oversight of an executive team rests on full transparency. Even today, a board cannot follow a CEO’s decision path in real time. Bias, intuition, informal channels, and political considerations are part of every leadership decision and only partly reconstructable. Oversight has therefore always worked through proxy mechanisms: regular reporting, an internal control system, internal audit, external review, risk committees, minute-keeping obligations.

These mechanisms hold up because a human decision-maker can be required to explain. They can lay out their reasoning after the fact, disclose assumptions, name discarded alternatives. Oversight, in other words, does not rest on direct analysis of thought and decision processes. It rests on accountability and the ability to demand disclosure.

With AI, the opacity is structurally different

For neural networks, this mechanism does not work the same way. The information is not stored as a readable set of rules. It is distributed across millions or billions of weighting parameters. It can only be inferred indirectly, by observing behaviour against defined test patterns. The dependencies between input and output are non-linear, and small changes in input data can produce markedly different results without any intuitive explanation being possible.

For systems that keep learning in operation, or that are regularly updated by the vendor, the problem intensifies further. Even a complete review as of a given date would already describe a different system after the next update. The results are reproducible; the path to those results remains largely opaque to the person exercising oversight. They can ask what the system decided, but only to a limited extent, why.

The difference from classic management oversight is not one of degree. It is structural. For a human CEO, the duty to explain is a lever that works because an explanation is fundamentally possible. For a neural network, there is no addressee for that duty. We cannot question an AI or put it on the witness stand.

The paradox and its consequence

This produces a situation I call the oversight paradox. The board carries a legal duty of oversight over systems whose internal decision logic it cannot, in principle, inspect to the same depth as the decision logic of an executive team. The established proxy mechanisms of classic management are not sufficient for this.

The consequence is sober. The board is not liable for the AI’s behaviour as such. It is liable for failing to organise adequate oversight of the AI. This shift does not change the law. It changes the benchmark for what counts as careful organisation. Oversight requires the power to intervene and the power to demonstrate. AI reduces both, unless governance keeps pace.

Not a new problem, but an unsolved one

The task of overseeing a complex system whose internal logic the overseer does not fully penetrate is not new. In aviation, pilots have relied for decades on avionics whose internal calculations they cannot follow in real time. In modern vehicles, electronic stability programmes and driver-assistance systems make decisions that the driver neither sees nor reviews.

The decisive difference lies in the response. Both industries have spent decades developing mechanisms that make oversight possible without full transparency: certification processes, redundancy requirements, independent review bodies, and standardised test procedures in aviation; functional safety under ISO 26262 (Road vehicles – Functional safety) and ADAS certifications (Advanced Driver Assistance Systems) in the automotive industry. In both sectors, it was accepted that direct inspection is not possible, and this was replaced with structured proxy mechanisms that are more formal and stricter than classic management reporting.

In corporate governance, comparable mechanisms for AI systems are still largely absent. Swiss company law provides the duty framework, not the control design for AI. The EU AI Act goes further and delivers compliance artefacts that can support proxy oversight: risk classification, technical documentation, defined roles, and requirements for record-keeping and traceability. It does not, however, translate into a ready-made board agenda. The board must extend its organisational regulations and its internal control system so that AI risks are covered.

The vendor reality makes it harder

In practice, few AI systems are built in-house. Most are components in a supply chain in which models, data, and training procedures sit with third parties. Anyone using a proprietary model from a hyperscaler or a specialised provider has no access to training data, model weights, or validation protocols. Oversight rests on what the vendor discloses voluntarily or contractually. Contracts without information, testing, and audit rights are a significant governance risk in this environment, as is the absence of a prepared plan for switching vendors should those rights prove unenforceable.

Four pillars of effective proxy oversight

Drawing on the experience of other industries and the regulatory anchors named above, four requirements can be derived that the board should embed in its governance. They scale by risk: a recommendation system for internal efficiency needs less depth than a model that makes automated decisions with external effect.

Performance and function monitoring: where the decision path cannot be inspected, the outcome must be measured continuously against defined criteria: hit rates, error rates, the distribution of outputs over time, harm indicators for safety- or reputation-critical applications. Without such metrics, any claim about an AI system’s effectiveness remains an assertion.
Independent review and enforceable vendor obligations: a review conducted by the same unit that operates or procured the model is not enough. An internally independent body is needed, and for higher-risk cases, an external review as well. Vendor contracts must include information, testing, and audit rights, supplemented by notification duties for model changes and security incidents, as well as exit options should those rights prove unenforceable.
Lifecycle control: output monitoring alone is blind to changes affecting the system itself: data drift, vendor model updates, changes to prompts or policies in language models, new data sources. Defined approval processes for such changes are needed, along with re-validation after updates and a technical and organisational ability to roll back. Without this discipline, the system under oversight changes quietly and invalidates every prior review.
Escalation and a named final decision-maker: oversight without powers of intervention is mere observation with no consequences. Defined thresholds are needed above which a human decision becomes mandatory, clear escalation levels, and, for emergencies, a documented shutdown criterion. Three questions must be answered organisationally: who may override, who must override, and who carries accountability, both for the override and for the consequences of the system’s recommendation. Without this separation, accountability blurs exactly where it is meant to attach.

What remains

The oversight paradox is not resolved by more technical understanding at board level alone. Nor is it resolved by forgoing AI, because competitive and regulatory logic in many industries is already moving towards AI-supported processes.

It is resolved by a governance approach that accepts direct inspection must be replaced by structured demonstrability, and that documents that evidence so it holds up in a dispute.

Art. 754 OR holds board members personally liable when they breach their duties. That duty does not change because of AI. What changes is the benchmark for careful organisation. It requires additional evidence, formalised testing, and robust escalation rules. Aviation has shown that effective oversight is possible even without full transparency, provided clear standards, robust metrics, and unambiguous escalation mechanisms are in place. That same discipline is now needed in corporate governance.

Sources

Floridi, L. (2021): The European Legislation on AI: A Brief Analysis of its Philosophical Approach. Philosophy & Technology.
Matthias, A. (2004): The Responsibility Gap: Ascribing Responsibility for the Actions of Learning Automata. Ethics and Information Technology.
Simmler, M. & Markwalder, N. (2017). Roboter in der Verantwortung? – Zur Neuauflage der Debatte um den funktionalen Schuldbegriff. Zeitschrift für die gesamte Strafrechtswissenschaft, 129(1), 20–47.