The wrong question

Posted on: 29 May 2026

A debate aired on the BBC last night on the regulation of artificial intelligence. Two guests, two positions, each internally coherent: regulate more, regulate less. The presenter handled the exchange with the professionalism one expects from public service broadcasting, distributed speaking time fairly, closed with the ritual observation that viewers should draw their own conclusions. Only one thing was missing: the right question.

The whole discussion ran along the axis of how much to regulate. Neither of the two interlocutors, nor anyone I have heard on the subject in the past eighteen months, raised the prior question: how do you regulate something you do not understand, and which by the time you do understand it will have mutated to the point where the rules you are now drafting are already obsolete. This is not rhetorical provocation, it is an epistemological problem that should precede any technical discussion of regulation, and which is systematically avoided because it destabilises both positions in the conventional debate.

Those in favour of regulation cannot address the question because admitting that the instrument is structurally lagging the phenomenon delegitimises their political stance. Those against do not want to address it because their thesis remains stronger when the conversation stays on how much rather than shifting to what. The result is a debate that repeats itself identically in newspapers, on television, at international summits, and that never approaches the structural knot.

The pattern is not new, and banking regulation provides the most visible clinical case. In the years after 2008 the supervisory authorities built Basel III on the assumption that banking risks could be modelled with enough precision to allow adequate capital requirements. While those requirements were being negotiated, calibrated and implemented, the financial system had already shifted a significant share of its risk outside the traditional banking perimeter, into private credit vehicles, leveraged lending funds, shadow banking structures that the framework designed for commercial banks simply could not see. The regulators were not incompetent. The real system had restructured itself faster than the regulatory machinery could adapt.

The same pattern recurred in telecommunications when voice migrated to IP, in retail when platforms bypassed the rules built for high street commerce, in financial markets when algorithmic trading rendered obsolete the control mechanisms designed around human contracting times. Each time the sequence is identical: rules are written for the objects of the previous moment, the objects of the next moment are left uncovered, political victory is declared on the basis of a map the territory has already stopped respecting.

The most instructive case, because the most visible from everyday experience, is KYC. Anyone who has opened a bank account in the last five years knows the routine: identity documents, proof of address, declarations on source of funds, professional activity questionnaires, periodic updates that require repeating the whole exercise at regular intervals. Banks apply the protocol with an intensity many customers experience as paranoid. Anyone who travels frequently, who handles legitimate financial flows across multiple jurisdictions, who simply lives a professionally articulated life, knows how much daily friction KYC produces. The question nobody asks publicly, but which anyone who knows the sector knows from direct experience, is whether all this apparatus actually solves the problem it claims to address.

The empirical answer is no. Money laundering continues to operate through structures that standard KYC does not see, because KYC is built on the typical customer, not the competent launderer. The serious operator moves through corporate vehicles designed precisely to be indistinguishable from the honest customer: consultancy firms with beneficial owners formally in order, property transactions with impeccable documentation, family offices structured in cooperative jurisdictions that produce all the paperwork the system asks for. Compliance does not catch laundering, it catches its amateur version, the kind the serious system abandoned twenty years ago. And here lies the clinical paradox: once the competent launderer has passed through KYC, the system has certified him as clean. The procedure that was meant to be an obstacle has become a legitimacy stamp.

The honest customer, meanwhile, experiences mounting friction. The bank queries every atypical transfer, blocks transactions falling outside expected patterns, demands additional documentation for operations that twenty years ago would have closed in two hours. The cost of friction is loaded entirely on the legitimate side of demand, while the illegitimate side has already paid the cost of constructing a façade that crosses the filter without setting off alarms. The system produces precisely the opposite of what it declares it wants to achieve.

There is a further layer worth naming with precision. Compliance has an institutional side effect that is not a bug but a feature: it transfers reputational risk from the regulator to the regulated. If a bank has executed every prescribed procedure and the customer turns out to have been a launderer, the bank states that the protocol was followed, and reputational risk dissolves into procedure. Had the regulator not imposed the procedure, the regulator would be the one answering politically for the failure. Regulation shifts responsibility along the chain, it does not reduce it. It produces systems in which everyone has done their job correctly according to the rules and the harm has nevertheless occurred. Nobody is at fault under the formal criteria. The real problem remains intact.

Apply this grid to artificial intelligence, and the pattern emerges. The AI Safety Institutes born out of the Bletchley summit in 2023 are building the equivalent of KYC for frontier language models. Pre-deployment evaluation protocols, compliance frameworks, audit metrics, certification criteria: the whole apparatus is designed to produce a fitness stamp on capabilities measurable with the tools available at the moment of evaluation. The tools available at the moment of evaluation are, by construction, those developed for previous models. The model evaluated today will be superseded in six months by one with emergent capabilities that today's evaluation could not even look for.

The large labs, from OpenAI to Anthropic to DeepMind, can afford this compliance. They have dedicated teams, internal red teaming infrastructure, the capacity to interface with official protocols, the resources to document conformity to the required criteria. Marginal actors and new entrants do not have these resources. AI regulation is reproducing exactly the KYC dynamic: the already large pass the filter with fixed costs absorbed by scale, the new or small are pushed out of the market not because they are less safe but because they are less structured to document their safety. The systemic problem, capabilities that mutate post-deployment, emergent interactions between different models, disintermediated uses that bypass official evaluations, remains entirely outside the perimeter.

The semantic metamorphosis of the last twenty-seven months is the most visible symptom of this structural difficulty. In November 2023 the Bletchley summit was called the AI Safety Summit, in May 2024 the Seoul summit retained the Safety label, in February 2025 the Paris summit became the AI Action Summit, in February 2026 the Delhi summit was the AI Impact Summit. The word safety has been progressively expelled from official vocabulary. In parallel, in February 2025, the UK AI Safety Institute was renamed the AI Security Institute, and its American counterpart became the Center for AI Standards and Innovation. The vocabulary change is not cosmetic. Safety implies a constraint on one's own conduct, security implies a defence against the conduct of others. A single word moves the perimeter of intervention from the mirror to the window. Regulators did not change laws, they changed vocabulary, and in doing so they tacitly admitted that the safety framework built in 2023 was already obsolete with respect to capabilities that had since mutated.

Popper would have named the problem with precision. A rule is effective only if it is falsifiable through observable enforcement. If you lack a model of the system sufficient to separate the contribution of the rule from the endogenous evolution of the phenomenon, you are not regulating, you are documenting. Taleb would reach the same conclusion by a different route: opaque systems are not governable through ex ante approval frameworks, they are governable only through ex post architectures of responsibility that bind whoever produces the risk to the consequences of the risk produced. AI safety regulation attempts the opposite: to establish ex ante what is acceptable in a system where the very category of acceptable mutates with the capability of the models.

The BBC debate closed, as usual, with the balanced observation that a middle ground will need to be found between the two positions. The middle ground between a structurally wrong position and a structurally wrong position is not the right position, it is the admission that the prior question was not asked. The question is what it means to regulate a system that mutates faster than the regulatory cycle, and what architecture of responsibility can function when understanding of the phenomenon is structurally behind the phenomenon itself. As long as the public debate stays on the axis of how much, AI regulation will continue to produce its equivalent of banking KYC: substantial friction on honest subjects, procedural cover on real risks, fitness certification for those who should have warranted scrutiny, market exclusion of those who deserved attention.

It remains to be asked who benefits from the prior question not being asked. But that is a conversation for another piece.