AI Training Programmes That Changed Behaviour, Not Just Completion Rates

Most AI training programmes produce a completion certificate and no measurable change. Dose is part of the difference. In BCG's 2025 survey of more than 10,600 employees, 79% of those who received over five hours of AI training were regular users, against 67% below that line. The programmes that changed behaviour were attached to a specific workflow and measured against a baseline recorded before anyone was trained.

What AI fluency means once you have to measure it

Fluency is a slippery word. Vendors use it to mean familiarity with a product. Learning teams often use it to mean a completed module. Neither definition produces something you can observe six weeks later.

The most useful public definition breaks the skill into four parts. Anthropic's AI Fluency framework, built with Rick Dakan of Ringling College and Joseph Feller of University College Cork, names them delegation, description, discernment and diligence. Delegation is deciding what the model should do at all. Description is communicating the task. Discernment is judging what comes back. Diligence is using it responsibly and owning the output.

This post is written for the person who owns delivery, not the person who owns the licence spend. That is the reader who has to answer for whether the team's output changed, and who therefore needs a definition that survives contact with a metric.

The competency most programmes skip

Nearly every AI training session covers description. Prompting demonstrates well on a screen, it produces an immediate result, and it fills an hour comfortably.

Discernment is the one that gets cut. Judging whether an output is correct, complete and safe to send is slower to teach, harder to script and impossible to demonstrate with a single clever example. It is also the competency that stops a wrong number reaching a client.

I would build a programme backwards from discernment. Give people 40 real outputs from their own workflow, half of them subtly wrong, and have them mark the set. That exercise generates a score you can repeat later, which is more than most curricula manage. The same logic applies when you are hiring, which is covered in the piece on interview questions that test AI fluency rather than tool familiarity.

The five-hour threshold and what it actually buys

BCG has run an annual employee survey on AI at work since 2023. The 2025 edition covered more than 10,600 respondents across 11 countries, which makes it the largest repeated read on this question in public.

Its finding on training is specific. Regular usage is sharply higher for employees who receive at least five hours of training and have access to in-person sessions and coaching. Reporting on the survey put the split at 79% regular users above five hours against 67% below. In the same survey, 18% of people who described themselves as regular AI users said they had received no training at all.

Twelve points is a smaller effect than it sounds

Read the bars honestly. Moving someone from four hours of training to six hours does not turn a refuser into a power user. It moves the odds by roughly 12 points, in a self-reported survey, with no control for the kind of organisation that bothers to fund six hours.

The 2026 edition sharpens the point. Across roughly 12,000 respondents, 72% said skill expectations had shifted and only 36% felt adequately upskilled. That is a supply problem in hours, and hours are the cheapest thing to fix. The reason most programmes still fail is not the hour count.

My position is that the five-hour figure is a floor to clear, then ignore. Spend the design effort on what those hours contain and what happens in the eight weeks afterwards. The gap between people who feel confident and people who actually use the tool is examined further in the analysis of the gap between reported trust in AI tools and reported usage.

Two government trials, one tool, two different answers

The clearest natural experiment in this field is public and largely unread. In late 2024 the UK government deployed the same AI assistant twice, under two different evaluation designs, and published both.

The cross-government experiment run by the Government Digital Service put 20,000 licences across 12 organisations and reported an average saving of 26 minutes per user per day. Satisfaction averaged 7.7 out of 10 and 82% of users said they would not want to return to working without it.

HMRC evaluated the same tool over the same months, and reached a different number.

Neither evaluation used a control group. Both rely substantially on self-reported time savings. The HMRC report applies an explicit downward adjustment for non-users and survey response bias; the cross-government report does not.

Why the two numbers differ

HMRC applied a stricter method. It reduced its survey figures by around 20% to account for people who held a licence and did not use it, and for the tendency of survey respondents to be enthusiasts. It also reported that 46% of non-users cited security and data privacy concerns, which is a finding no enthusiasm-weighted average will surface.

The cross-government report contains the sentence that matters most for training design. It found a correlation between a user's confidence and familiarity with AI and the time they saved, and concluded that this highlights the need for change management programmes covering communications, engagement and training.

That is a correlation, not a causal claim, and the report is careful to say so. It still points at the practical question. If confidence predicts saving, the programme's job is confidence in a specific task, not awareness in general. Applying HMRC's 20% haircut to any vendor-supplied saving figure is the single cheapest piece of discipline available, and it belongs alongside the tests set out in the forensic read of published AI case studies.

Training pays most where the skill floor is lowest

Two studies with real experimental design tell you where to spend a limited budget, and they point the same way.

The first is Generative AI at Work, published in the Quarterly Journal of Economics in May 2025. Brynjolfsson, Li and Raymond studied the staggered rollout of an AI assistant to 5,179 customer support agents. Access raised issues resolved per hour by 14% on average. For novice and lower-skilled workers the gain was 34%. For experienced, high-performing agents it was minimal.

The mechanism the authors describe is the interesting part. The tool spread the practices of the best agents to everyone else, moving new staff down the experience curve faster. That is a training effect delivered by a product rather than a course.

The teamwork result changes who you enrol

The second study is The Cybernetic Teammate, a pre-registered field experiment with 776 professionals at Procter and Gamble, run in summer 2024 and later published in Organization Science. Participants worked on real product development problems, randomly assigned to work alone or in pairs, with or without AI.

Individuals working with AI matched the performance of two-person teams working without it. The AI also cut across functional silos: without it, research staff proposed technical solutions and commercial staff proposed commercial ones, and with it both groups produced more balanced proposals.

The practical read for a training programme is uncomfortable. If one trained person with a model performs like an untrained pair, the unit of enrolment should not be the department. It should be the workflow that currently needs two people to cover two kinds of expertise.

What the UK evidence review found actually works

In June 2026 the UK government published the output of its Skills for AI programme, What Works for AI Upskilling in the UK. It draws on 23 workshops with around 150 organisations, 10 case studies and a UK employer survey of 536 responses.

Two findings are worth acting on immediately. Over 44% of surveyed organisations report using AI tools daily, while most remain early in adoption, and staff largely learn through trial and error, peer support, online videos and prompts built into the tools. Formal training is not where the learning is happening.

The barriers are also not what training vendors assume. The review names limited time, staff pressure, cost, unclear provision and fear of failing at technical material. Motivation is not on the list. If your programme's core problem statement is engagement, you are solving for something the evidence does not show is broken.

Short and stackable beats a day off site

The review's design guidance is summarised as PRIMES: practical, reachable, integrated, modular, expandable and sustainable. Stripped of the acronym, the operative recommendation is that very short modules of 30 to 90 minutes are more practical than longer courses.

That directly contradicts the standard corporate shape, which is a half day with a vendor and a follow-up email. It also fits what the five-hour threshold implies: five hours accumulated across six weeks in the flow of work, not five hours in a room.

The last row is the honest split. Run both. The mistake is running only the first and describing it as capability building.

Most programmes measure the wrong four things

Learning teams have had a four-level evaluation model since the 1950s: reaction, learning, behaviour, results. Reaction and learning are cheap to collect. Behaviour is the level everyone says they want and almost nobody instruments.

AI training makes this worse, because the tools generate usage telemetry that looks like behaviour data and is not. Weekly active users tells you a licence was opened. It does not tell you a workflow changed.

The time saved has nowhere to go

Here is the finding that should reshape the closing session of every AI programme. BCG's 2026 survey found that 66% of regular AI users receive limited or no guidance on what to do with the time they save.

Twenty minutes returned to an unstructured day disappears without trace. Twenty minutes redirected into a named backlog, a second review pass or one more client conversation shows up in a number somebody already reports. The redirection is a management decision, and it is the step most programmes leave undesigned.

Budget pressure makes this urgent. The Wharton Human-AI Research and GBK Collective survey of more than 800 senior leaders, published in October 2025, found training investment slipping by 8 percentage points year on year and confidence in upskilling programmes down 14 points, even as 46% named delivering effective training as a top barrier. Programmes that cannot show a result are losing their funding, which is the correct outcome for programmes that cannot show a result. Function-level payback estimates are worked through in the breakdown of where AI payback appears by business function.

Where this argument is weakest

This section exists because the evidence base here is thinner than the confidence with which it is usually quoted, including by me.

The most quoted number in learning and development is not evidence

The claim that only 10% of training transfers to the job appears in hundreds of articles and several academic papers. It traces back to a 1982 article by David Georgenson in the Training and Development Journal, and Robert Fitzpatrick's examination of the original found that Georgenson used it as a rhetorical opener, quoting what training directors say, with no evidence or authority cited for the figure.

I nearly opened this post with that number. It is the exact failure the sourcing rules exist to catch: a striking statistic restated so often it acquires the texture of a finding. Six restatements is one source, and in this case the one source was a rhetorical question. The figure should be retired.

Almost everything here is self-reported

Of the six main sources in this post, only the Quarterly Journal of Economics study and the Procter and Gamble experiment have anything resembling a control. The BCG surveys, the UK employer survey and both government evaluations rest on people estimating their own behaviour and their own saved time.

Self-reported time saving is the least reliable measure in the set. People who like a tool overestimate; people who never opened it do not respond to the survey. The 2.2 times gap between two UK evaluations of the same product is the size of that problem, measured. Treat every number in this post as directional, and treat your own baseline as the only figure you can defend.

A 90-day design that produces evidence

The following is deliberately narrow. It trains fewer people than a company-wide rollout and produces something a finance director will accept.

Weeks 1 to 4 are the baseline. Pick one workflow with a number the business already tracks, and record that number weekly with nobody trained. Cases closed, drafts produced, cycle time, first-response quality. Do not announce the programme yet, because announcing it moves the baseline.

Weeks 5 to 10 are the cohort. Six sessions of 60 to 90 minutes, one per week, each built around the team's own live work rather than demonstration examples. Total contact time lands above the five-hour threshold. One session is the marked discernment exercise. One session is the redirection decision: what the recovered time is now for, agreed with the manager who owns the workflow.

Weeks 11 to 13 are the measurement. Same metric, same collection method, no change to definitions. Report the delta, the number of people who used the tool in that workflow in week 12, and the honest confound list. If the cohort's manager also changed something else, say so.

Two rules make this hold. Cap the cohort at one team, because a mixed cohort gives you a mixed baseline. And do not let the programme own the metric. If the workflow's own manager does not already report that number, pick a different workflow. The failure patterns that follow from a central team owning delivery are set out in the analysis of when an AI centre of excellence becomes the bottleneck, and the quieter forms of refusal are covered in the piece on passive resistance to AI rollouts.

Frequently asked questions

Where to start this week

Open your last AI training deck and find the slide that names a workflow. If there is no such slide, that is the finding, and it explains the missing behaviour change without any further analysis.

Then pick one team and start the four-week baseline before you book a single session. Recording a number you already collect, for a month, with nobody trained, costs nothing and is the only thing that will let you answer the question you will be asked in the next budget round. Performance expectations that follow from all this are worked through in the piece on rewriting review criteria for AI-assisted work.

Related reading