Part 5 — Measure What Happened
The Dashboard and the 30/90/180-Day Checkpoints
Almost nobody does this part. A business case gets approved, the project ships, and six months later everyone has a general sense of whether it went well and nobody has the number. This piece closes the loop the other four open: it turns the projection in Part 4 into a measured result, with a real decision attached to it on a fixed schedule.
Set the dashboard before launch, not after
Pull the metrics directly from the two blocks you already committed to:
- The KPI baseline from the impact chain in Part 2 — the number, and who agreed it
- The financial target from the four-step calculation in Part 3 — the conservative-case benefit and the payback date from Part 4
A working dashboard for a mid-sized company doesn’t need more than a handful of numbers, tracked consistently:
- The business KPI itself (conversion rate, cost per transaction, error rate — whichever one the project was built to move)
- Adoption — the metric most business cases skip, and the one most likely to turn a good projection into a disappointing result. Active users or usage rate, measured weekly from week one, not assumed.
- Quality or error rate on the AI-handled share of the work
- Cost — actual spend against the budgeted figure from the business case, direct and indirect
- Volume or throughput — is the process actually handling what was projected, or has scope quietly narrowed
Track them on a fixed cadence — weekly for adoption in the first month, monthly for everything else — for the first six months. A number checked once at the end tells you far less than the same number watched as it moves.
Three checkpoints, each with a real decision
This is the part that makes a dashboard useful rather than decorative: a decision attached to each date, agreed before launch, not improvised afterward.
Day 30 — Is it being used? Too early to judge financial return. Not too early to judge adoption. If usage is well below plan at day 30, that’s the moment to intervene — more training, a narrower first use case, direct involvement of the people expected to use it — not day 90, when the pilot budget is half spent and sunk-cost thinking sets in. Decision: continue as planned, or adjust the rollout.
Day 90 — Is the pilot hitting the numbers from the business case? This is the point the “decision point” in Part 4’s implementation plan usually refers to. Compare actual KPI movement and actual cost against the conservative case you presented — not the optimistic one, and not a revised-down version of the target quietly substituted in because the real number came up short. Decision: proceed to full rollout, extend the pilot, or stop.
Day 180 — Did it pay for itself, and does the KPI hold? Enough time for the payback calculation from Part 3 to be tested against reality, and enough time to see whether an early KPI improvement has held or faded as novelty wore off. This is also the point to revisit the risk register from Part 1 — has the adoption risk actually materialized, has anything on the legal or data side changed. Decision: scale the project to its full intended scope, keep it at current scope, or wind it down.
What to do with a result that misses
A miss isn’t a failure of the method — it’s the method working. The entire point of setting a baseline and a conservative case up front is to be able to say, cleanly, that a project underperformed, and why: adoption came in lower than modeled, the error rate in production didn’t match the pilot, the volume assumption was wrong. That’s a specific, useful sentence. “It didn’t really work out” six months after an unmeasured launch is not — and it’s the outcome this whole playbook exists to prevent.