AI Centre of Excellence: Accelerator or Bottleneck? Measure Decision Latency

76% of organisations now have a Chief AI Officer, up from 26% a year earlier. In the same survey, 79% of executives said they are decentralising decision-making. An AI centre of excellence exists to hold both facts at once. The structure is not the decision. Which decisions it keeps is the decision.

What an AI centre of excellence actually owns

An AI centre of excellence is a central team that owns AI standards, tooling, model access, evaluation criteria and risk review. Business units own the workflows and the outcomes. It accelerates delivery when it removes repeated work. It becomes a bottleneck when it also owns approval for every use case.

Most enterprises arrive at this question backwards. They ask whether to build a centre of excellence, then work out what it should do. The better order is to list the decisions that AI has created, then ask which of them a business unit can safely get wrong.

That reframing does most of the work. A decision whose failure mode is a wasted sprint belongs near the work. A decision whose failure mode is a regulator, a customer contract or an irreversible action belongs somewhere accountable and singular.

The four jobs a hub is given

BCG's August 2026 description of the centralised AI hub gives it four roles: coherence, speed, governance and adoption. Coherence means one direction and no duplicated builds. Speed means shared components, so each unit does not solve the same problem twice.

Governance means risk, legal and security are present at the first design review rather than the last. Adoption means the behaviour change that the technology does not deliver by itself, which is the part most programmes underfund.

Hub size in BCG's client base runs from about 10 full-time staff to more than 100, and there is no standard reporting line. Hubs report to the CIO, the chief strategy officer, the CFO, the chief HR officer or the chief executive, depending on where the firm believes the constraint sits.

That variation is itself a finding. If the structure were solved, the reporting line would have converged by now. It has not, which tells you the answer is contingent rather than universal.

Two org charts moving in opposite directions

IBM's Institute for Business Value surveyed 2,000 chief executives and equivalent leaders across 33 geographies and 21 industries between February and April 2026. Two findings from that survey sit oddly together.

The first is that 76% of surveyed organisations now have a Chief AI Officer, up from 26% a year earlier. The second is that 79% of executives confirmed they are decentralising decision-making and distributing accountability as AI spreads across the enterprise.

A single company can be doing both at once. It only reads as a contradiction if you treat decisions as one undifferentiated pile, which is exactly the mistake that produces a bad operating model.

Authority is concentrating

The clearest published example of deliberate centralisation is not a company at all. It is the US federal government. Memorandum M-25-21, issued by the Office of Management and Budget in April 2025, required covered agencies to designate a Chief AI Officer within 60 days. It also required them to convene an agency AI governance board within 90 days.

You can check that this happened rather than take it on trust. The Federal Reserve published its own compliance plan naming the structure it stood up. Whatever you think of the policy, it is a clean statement of what centralisation is for: a named person, a standing board, and one inventory.

The private sector version shows up in BCG's finding that 73% of chief executives report having the final say on AI decisions, roughly double the prior year. Final say is not the same as doing the work. It is the reservation of a small set of decisions to the top of the house.

Execution is dispersing

The counterweight is BCG's long-running 10-20-70 split: 10% of AI value comes from algorithms, 20% from data, and 70% from changes to the operating model. Operating model change is local by definition. It happens in a claims team, a collections team, a support queue.

No central hub can rewrite 40 workflows it does not run. This is why the mature answer is a split of decision rights rather than a split of headcount. It is also why the question of who owns an agent once it is live matters more than where the builders sit.

The governance scaffolding is also spreading faster than the delivery capability. Stanford's 2026 AI Index reports that the share of businesses with no responsible AI policy fell from 24% to 11%, and AI-specific governance roles grew 17% during 2025. Policies are cheap to write. Reviews are not.

What centralising actually buys you

The honest case for a central function is narrower than the pitch decks suggest. It is not that central teams build better models. It is that they produce a single, defensible record of what was built, tested and approved.

Grant Thornton surveyed 950 business leaders across 10 industries between 23 February and 18 March 2026. 78% lacked strong confidence they could pass an independent AI governance audit within 90 days. That is the baseline the whole debate sits on.

Audit readiness looks like a step function

Inside that survey, the spread by maturity is the interesting part. Among organisations still piloting AI, 7% were very confident of passing that audit. Among organisations with AI fully integrated, 74% were. The same report describes this as a tenfold difference.

Be careful what you conclude. This is a correlation between integration maturity and confidence, not proof that a centre of excellence caused either. Firms that integrated AI deeply are also firms that invested in evidence, and the survey cannot separate those two things.

The board-level numbers in the same study point at the gap a central function is meant to close. 75% of boards had approved major AI investments. 48% had not set AI governance expectations, and 46% had not built AI risk into ongoing oversight. Money moved before oversight did.

The risk side is sharper still. 72% of leaders said they are giving agentic AI access to their data and processes, while 20% have a tested AI incident response plan. Stanford's Index counted 362 documented AI incidents in 2025, up from 233 in 2024. If you want a structured way to see where your own programme sits, the agent governance maturity model is a more useful lens than a headcount plan.

The compliance clock moved, it did not stop

A common argument for delaying the governance build is that Europe blinked. That reading is wrong in a way that matters for how you staff.

The EU AI Act's high-risk obligations were deferred by the 2026 omnibus agreement. Obligations for stand-alone Annex III systems moved to 2 December 2027, and for high-risk AI embedded in regulated products under Annex I to 2 August 2028. Those are real extensions and they are substantial.

What did not move is as important. The Article 4 AI literacy duty has applied since 2 February 2025. General-purpose model obligations have applied since 2 August 2025. The Article 50 transparency rules landed on 2 August 2026, with a short grace period for the watermarking provision. A checklist for that last set is covered in the EU AI Act transparency requirements breakdown.

The practical reading is not to relax. It is that you now have roughly 15 months to build the evidence system that the December 2027 date will test. Building that twice, once per business unit, is the expensive way to do it.

Where the bottleneck appears, and what it looks like

A centre of excellence rarely fails on capability. It fails on queue time. The failure is arithmetic, not politics.

A central review team has fixed throughput in any given quarter. Business demand for AI does not. Once the number of waiting requests exceeds what the team can process, the queue stops being an administrative detail. It becomes the company's actual AI policy, set by whoever happens to be at the front of it.

BCG names this failure mode explicitly in the same article that argues for the hub. Starting in a highly decentralised way creates fragmentation, it says, and waiting too long to distribute ownership creates a central bottleneck. Both failures are described as pitfalls of the same design.

My position is that a centre of excellence should not own approval and standards simultaneously for more than about a year. Standards are a permanent central asset. Approval is a temporary one, held only until the units can apply the standard themselves, and the handover date belongs in the founding charter rather than in a negotiation later under pressure.

Blocked demand does not disappear

This is the part that makes the bottleneck expensive rather than merely slow. A team told to wait six months for approval on a summarisation workflow does not wait six months. It uses a tool that does not require approval.

The result is that heavy central control can increase ungoverned AI use rather than reduce it, because the control only binds the people who ask. The approval path is the product here, and treating the internal approval path as something to design is the difference between governance and theatre.

Gartner has forecast that more than 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value and inadequate risk controls. Note that the list contains both failure directions. Weak controls kill projects, and so does cost, which central bureaucracy adds.

What a federated model actually fragments

The case against federation is usually made as a vague worry about standards. It is more specific than that, and BCG's separate August 2026 piece on governing agents at scale sets out the mechanism.

Managing AI platform by platform, it argues, duplicates effort, escalates cyber risk and complicates cost control. Each business unit rebuilds identity handling, logging and approval, slightly differently. Nobody holds a single inventory of what is running.

The recommended answer is an enterprise-wide control layer rather than per-platform governance, with a reported effect of cutting setup for a new agent from weeks to about a day once a standard path exists. That is a speed argument for centralisation, which is the opposite of how the debate is usually framed.

The evaluation standard is the load-bearing one

If you federate only one thing badly, make sure it is not evaluation. Without a shared definition of good enough to ship, every unit sets its own threshold and none of them are comparable.

That stays invisible until an incident, at which point you discover that one team tested against 40 hand-written examples and another against a versioned suite. The cheapest central asset in the whole programme is a shared harness, and building the evaluation suite once is the piece I would centralise before anything else, including model access.

The conditions that pick the structure for you

The consensus answer is hub and spoke. It is correct and almost useless, because it names a shape without naming a split. Two firms can both run hub and spoke and have completely different bottlenecks.

Start instead with the decision inventory. The table below is the split I would defend, and the test column is the part to argue about, not the placement.

This split is an editorial position, not a survey finding. It is drawn from how the sources above describe the two failure modes, and reasonable operators put the last row entirely on the central side.

Where this argument is weakest

Three problems with everything above, including the parts I believe.

Nobody has published the latency numbers

The whole bottleneck case rests on queue time, and no survey in this field measures it. There is no published dataset of median days from AI use-case request to decision, broken down by operating model. Until somebody collects it, the bottleneck argument is mechanism plus anecdote, not evidence.

That is a real weakness and it cuts both ways. The case for centralising is equally unmeasured. Grant Thornton measures confidence in audit readiness, which is a perception, and IBM measures what executives say about their own org design.

The phase models are consultancy artefacts

BCG's four-phase evolution, from hub-led to centre-and-pod to business-unit-led to an AI-first organisation, is a clean story. It is also the kind of story that gets constructed after the fact from client engagements, not derived from a tracked cohort.

Treat it as a useful vocabulary rather than a forecast. No published data shows that firms actually pass through those phases in order, or that the ones that do outperform the ones that do not.

The competing view deserves stating plainly. A serious operator could argue that the operating model is downstream of talent supply, and that firms centralise simply because they cannot hire enough AI engineers to staff every unit. On that reading, structure is a symptom and this whole debate is misdirected.

Frequently asked questions

Where to start this quarter

Two moves, both cheap, both diagnostic rather than structural.

First, instrument decision latency before you reorganise anything. Pull the last 20 AI requests that went to a central team and record two dates for each: when it was raised, and when it got a decision. If the median is under three weeks, your centre is not the constraint and reorganising will not help.

Second, write the sunset clause now. Put a date in the charter on which approval rights transfer from the centre to the business units. Name the specific condition that has to be true by then, such as a published evaluation standard the units can run themselves. A hub that has never written down when it stops deciding is a hub that never will.

Related reading