Playbook 1 — The Adoption Gap
Why a working pilot and an adopted pilot are two different projects
A tool that passes testing and a team that uses it daily are not the same outcome. Most AI budgets are spent as if they were.
Here is how the gap opens. A pilot is scoped to answer a technical question: does the model produce good output, does it integrate, does it hold up on real data. It succeeds on those terms, gets signed off, and everyone moves on. The measure that mattered — whether people changed how they work — was never the one the project reported on. So a green light on the tool reads as a green light on the adoption, and nothing about the tool tells you whether anyone opened it on Monday.
The change-management research firm Prosci publishes a figure that most AI failures trace to human factors rather than the technology itself. Read it directionally, not as a precise number: Prosci sells change-management services, so the framing is not neutral. But it matches what you see without a study. The tool rarely fails. The plan around the tool fails.
The gap has two edges, and they fail differently. The first is between approved and used: leadership signs off, licences are bought, usage flatlines. The second is between used and used correctly — someone opens the tool, gets a bad first output, and quietly goes back to the old way while the licence stays active. Only one of those looks like a problem on the invoice.
Licence counts hide both. A seat that is paid for and a seat that is exercised are identical line items. The honest signal is task-level: did this piece of work get done in the tool, or beside it.
What to settle before you call a pilot a success:
- The success metric includes a behaviour, not just an output.
- You know who is expected to use it, on which task, how often.
- Someone owns adoption, separately from whoever owns the tool.
- You have a baseline for how the task was done before.
- There is a defined point where “nobody is using it” counts as a result.
None of this is about enthusiasm. It is deciding, before the pilot starts, that a tool nobody uses has failed — even if it works perfectly.
Next: Playbook 2 names the two questions a rollout has to answer before day one.