Fund a testable use case with an owner, a baseline and a clear stop condition.
“We need AI” is too broad to scope. Name the work that should improve, the people affected and the decision the project will support. A useful opportunity statement is narrow enough to test: “Reduce the time staff spend preparing answers to recurring policy questions while keeping source access and final approval in place.”
Define the outcome and baseline
Choose a process owner and one measure that reflects the desired result: turnaround time, rework, quality, staff effort or customer experience. Record the current range, not just an average. A fast result that increases correction work is not an improvement. Agree what success looks like and what level of error is unacceptable.
Work: staff answer recurring policy enquiries. Baseline: time to a reviewed response and correction rate. Proposed change: AI drafts from approved sources. Boundary: staff send the final response. Decision: proceed only if quality and effort both improve.
Check the dependencies the demo hides
Walk through real cases and ask where the source information lives, who may see it, which system must be updated and who handles exceptions. A process that relies on informal knowledge or inconsistent data definitions may need cleanup before an AI layer. Check integration constraints, privacy requirements, procurement and ongoing costs early enough to change the scope.
Use four lenses: process (repeatability and exceptions), data (quality and permission), technology (integration and support), and people (ownership and review). The NIST AI RMF frames this as understanding the intended use, context, benefits, costs and risks before measuring and managing the system.
Prioritise with evidence, not a score alone
Compare candidate use cases on potential value, feasibility and consequence of failure. A scoring table can start the conversation, but do not let a single number hide a critical dependency. A high-value use case with no approved data path is a discovery project, not a launch-ready project. A lower-value, well-bounded workflow may be the better first delivery.
| Question | Evidence to collect |
|---|---|
| Is there a real problem? | Case volume, delay, rework or unmet demand |
| Can the system act safely? | Permissions, exception rules and reversibility |
| Can we measure it? | Baseline, representative cases and acceptance criteria |
| Can we operate it? | Named owner, support path and budget |
Design a pilot that can say no
Build a representative evaluation set before implementation: easy cases, edge cases, missing information and cases where the correct answer is to defer. Decide who grades results, how disagreement is resolved and which errors block launch. The pilot should test the end-to-end workflow, not just response quality in isolation. AWS preproduction guidance describes combining offline evaluation, live testing and human feedback before production.
Write explicit exit decisions: proceed, revise or stop. If the pilot misses a threshold, find whether the cause is source quality, workflow design, model behaviour or an unrealistic use case. Do not label the entire programme a success because a few examples looked convincing.
Make ownership part of the investment
Before funding delivery, name who owns business outcomes, data access, technical operation, incident response and model or prompt changes. Agree usage and support budgets. The output of readiness work should be a short decision record: the opportunity, baseline, dependencies, risks, proposed pilot, acceptance criteria and owner.
Sources & further reading
External guidance informs this article; the examples and recommendations above are AI Dive’s interpretation for practical planning.