AI & Data
Why most AI pilots never reach production — and what the ones that do have in common
AI pilots are easy to start and hard to finish. The model is rarely the problem: data access, integration, ownership and operating costs are. Here is what separates the projects that ship from the ones that stay demos.
A pilot proves the idea, not the system
A pilot answers one question: can a model do this task well enough on a sample of data? Production asks many more. Can it get fresh data every day without someone exporting a spreadsheet? Can it plug into the system where people actually work? Who is accountable when it is wrong? What does it cost at full volume? Pilots stall because they were scoped to answer the first question and nobody planned for the rest.
Pick a problem with an owner and a number
Projects that reach production start with a business owner who wants the outcome and a metric they already track — hours spent on a task, error rate, time to answer a customer. That owner decides what "good enough" means and fights for the integration work. Projects started because a team wanted to try AI rarely have either, and they end when the demo does.
Fix data access before modelling
The slowest part of most AI projects is getting reliable, permissioned access to the data the model needs. In a pilot, someone copies a sample. In production you need pipelines that refresh on schedule, respect who is allowed to see what, keep personal information where Canadian privacy law requires, and alert when the data changes shape. Teams that build this first find the modelling straightforward; teams that leave it last find it is the whole project.
Design for being wrong
Every model makes mistakes, so production systems need a plan for them: confidence thresholds, a human review step for high-impact decisions, an easy way for users to flag bad output, and logs that let you investigate a complaint months later. These controls are also what auditors, privacy officers and customers will ask about. Building them in from the start is much cheaper than retrofitting them after an incident.
Budget for running it, not just building it
A model in production is a service. It needs monitoring for accuracy drift, a process to retrain or update it, capacity planning for compute (including GPUs for larger models), cost tracking per request, and an on-call owner. If nobody is funded to run it after launch, it will quietly degrade until someone switches it off. Treat operating cost as part of the business case from day one.
A practical path from pilot to production
Start with one workflow and one owner. Build the data pipeline and access controls first. Integrate the model into the tool people already use, behind a review step. Measure against the metric the owner chose. Then decide, with real numbers, whether to widen it. It is slower than a flashy demo, and it is the route that ends with something the business relies on.
Frequently asked questions
- How long does it take to put an AI pilot into production?
- It depends far more on data access and integration than on the model. A focused use case with clean, accessible data and one target system can move in weeks; one that needs new data pipelines and approvals across several teams takes months.
- Do we need GPUs to run AI in production?
- Not always. Many production uses call hosted models through an API or run small models on ordinary servers. Dedicated GPU capacity matters when you train or fine-tune models, run large models yourself, or need data to stay in a specific Canadian region.
- Who should own an AI system after launch?
- A named business owner who is accountable for the outcome, supported by a technical team that monitors accuracy, cost and availability. Without both, production AI systems tend to degrade unnoticed.
Related reading
Next step
Want early access to our thinking?
Subscribe to receive Aethon Core insights as they publish — practical, plain-language content on enterprise technology from people who build it.