Most things sold as AI problems are process problems with a model bolted on. The test I use, the shapes of work where the economics change, and why picking the right technique matters more than picking the model.
The demo is easy. Almost everything sold as AI automation demos beautifully, because a demo is a happy path with a human standing beside it explaining what you’re looking at.
Production is different. Nobody is standing there, the inputs are worse than the demo data, and being wrong has a cost that lands on someone. The gap between those two states is where an AI budget dies.
This is how I decide whether something is worth building.
One question comes before all the others: what does being wrong cost, and who finds out?
That sorts most of it.
If being wrong is cheap and immediately visible, a draft a human reads before it goes anywhere, a suggested category the operator can override, a first pass at a summary, then AI is a good fit and you should get on with it. Errors surface at once and cost almost nothing.
If being wrong is expensive and invisible, a number in a report somebody acts on, a compliance record, a price, then the model isn’t the hard part. The reconciliation is the hard part. If you can’t describe how you’d catch a wrong answer, you aren’t ready to build.
I’ve built this both ways. On a SaaS platform where the AI answers questions about money, every answer is tested against the underlying ledger to a tolerance of one penny on value and 0.1% on rate. A run that reports a gain where the ledger shows a loss is a hard failure, not a warning. That check is more engineering than the feature it protects, and it’s the only reason the feature is safe to put in front of someone about to act on it financially.
Three shapes repay the effort.
The strongest case and the most underrated. Most businesses sit on a pile of unstructured text or audio containing the answer to a question they already pay people to guess at.
In my own business we could see leads weren’t converting. We couldn’t see why, because the why was in thousands of recorded sales calls and onboarding notes no dashboard had ever read. A human could have listened to them. No human was ever going to.
That’s the shape: a question you already care about, with the evidence sitting in a format nobody can process at volume. The model being clever is beside the point. What changed is that reading all of it went from impossible to merely expensive.
Somebody types something into a chat window, an email or a form, and downstream a person retypes it into a system. That retyping is pure loss and it’s everywhere.
On one platform users submitted entries as ordinary sentences in a chat tool. Interpreting them, attributing them and reconciling the totals was manual, so the headline numbers the community relied on couldn’t be audited. Parsing that into a structured ledger, 79,000 records as of August 2026, saved more than the typing. It turned an unverifiable claim into a number with a trail behind it.
The economics are unusually good here because the alternative is not “a person does it slower”. The alternative is that nobody does it and everyone tolerates the uncertainty.
Some tasks have no right answer, only a consistent one. Grading the cosmetic condition of a used device is one. Two experienced people will disagree at the margins and that’s tolerable. The same person grading the same device differently on Tuesday is not.
For work like this a model beats a human because it doesn’t get tired. It’s no smarter. Design for repeatability: log everything, and treat a grading system that returns different answers for the same photographs as worse than no grading system.
When the process is the problem. If three people must approve something and each takes two days, the delay is the approvals, not the typing. Automating the typing gets you a faster route to the same queue. Fix the process, then see whether anything is left worth automating.
When the volume isn’t there. A workflow that happens eleven times a month won’t repay a build, a test harness, monitoring and somebody’s continuing attention. Do it by hand and spend the money on the thing that happens eleven thousand times.
When you can’t check the answer. Covered above, but it bears repeating, because it’s the one that does real damage instead of merely wasting money.
When it’s a search problem, a report, or a rule. A great deal of what gets specified as AI is a well-indexed search box, a query nobody wrote, or four lines of conditional logic. Those cost less to build, less to run, are easier to explain to an auditor, and need no re-evaluation every time a model version changes.
There’s a reflex in this market to reach for the biggest general-purpose model and call it done. It’s the most expensive habit in applied AI and it’s rarely the right answer.
You have more than one tool.
Prompting an API suits work where the model already knows the domain and you need language handled well. It’s the fastest thing to build and the easiest to get wrong at volume, because you pay per call forever.
Retrieval suits work where the answer lives in your documents and not in the model. You fetch the relevant history at query time and hand it over. The advantage people miss is auditability: retrieved context can be inspected, cited and corrected. Context baked into weights cannot.
Fine-tuning suits work where the domain has structure the model has never seen and can’t infer from a prompt. I fine-tuned a model on domain history for a prediction system, because a general model can describe that domain fluently and still can’t price it. Training on what happened is the step that moves an answer from plausible to usable, and it’s the step most projects skip.
MCP servers suit a problem I now see in every business I look at: staff use AI assistants daily, and with no access to company context the assistant invents one. I built MCP servers over my own business systems so a question about bookings, pipeline or one specific customer is answered from live data, with the permissions that come with it. Context is the whole problem in business AI use, and it’s a plumbing problem, not a model problem.
And sometimes none of them. A classifier solved twenty years ago, or a database query, will beat all four on cost, speed and explainability.
Knowing which of these a job wants is the expertise. The plumbing afterwards is comparatively easy.
A narrow, well-specified task, classify this, extract these fields, decide whether this transcript mentions X, doesn’t need a model that also writes poetry and passes a law exam. It needs the smallest model that clears the bar. The gap in running cost between those two choices isn’t a few per cent. At volume it decides whether the workflow pays for itself.
Two things follow.
Volume changes the architecture as well as the invoice. Per-call pricing is fine for hundreds of calls and painful for millions. Work sized for a smaller model, running on hardware you already own, converts a variable per-token charge into a fixed cost you have already paid. That’s what makes processing an entire back catalogue viable instead of theoretical.
Measure the choice, don’t assume it. I have yet to be shown this in a codebase I didn’t write. In my own pipeline the directional briefs run through two backends over identical input, one local and one cloud, producing a side-by-side comparison for a human to judge. The local model is in production because it was shown to be good enough for that job. Had it lost, it wouldn’t be there.
Do that once, properly, and you can stop guessing. Skip it and you’re either paying for capability you never use or shipping worse output than you could have had.
The build isn’t the expensive part. These are:
For a lot of businesses the sharpest constraint is neither cost nor capability. The data can’t leave.
That’s my situation. My business handles children’s data, so sending call recordings to a hosted model was never on the table. The sensitive stages run on hardware I own, speech-to-text and a large open-weight language model, with only structured conclusions and quoted evidence crossing the boundary. No model vendor sees the recordings and nothing trains on them.
This comes up more than people expect. Legal, healthcare, finance, education, and anyone whose data protection officer has already refused cloud AI. The useful thing to know is that the refusal is normally about one stage instead of the whole pipeline. Work out which stage can’t leave, run that stage locally, and the rest of the architecture opens back up.
Every AI project I’ve been called into had failed on judgement. The technology has been good enough for a while.
Keep reading
Day rates, retainers and what the money buys, with the sources named and my own rate card on the page instead of behind a call.
Read itWork with me
I reply within one working day to arrange a free 30-minute call.