Where AI automation pays, and where it does not

Almost everything sold as AI automation demos beautifully, because a demo runs the happy path with a human beside it explaining what you’re looking at. Production has none of that: nobody is standing there, the inputs are worse than the demo data, and being wrong has a cost that lands on someone.

That gap is where the money goes, and here’s how I decide whether something is worth building.

The test

What does a wrong answer cost, and who notices?

If being wrong is cheap and immediately visible (a draft a human reads before it goes anywhere, a suggested category the operator can override), AI is a good fit and you should get on with it.

If being wrong is expensive and invisible (a number in a report somebody acts on, a compliance record, a price), the check that catches it has to be designed before the model is. If you can’t describe how you’d catch a wrong answer, you aren’t ready to build.

Where the AI answers about money, I test every answer against the ledger before a user sees it; the SaaS case study has the tolerance.

The shapes that pay

1. Reading what nobody has time to read

I’d look here first. A business sits on a pile of unstructured text or audio containing the answer to a question it already pays people to guess at, and has done for years without anybody opening it.

Mine was one: the reason our leads weren’t converting sat in thousands of recorded sales calls that no dashboard had ever read. The case study has the result.

2. Turning free text into structure

Somebody types something into a chat window, an email or a form, and downstream a person retypes it into a system. That retyping is pure loss and it’s everywhere.

On one platform users submitted entries as ordinary sentences in a chat tool. Reconciling the totals was manual, so the headline numbers the community relied on couldn’t be audited. Parsing that into a structured ledger, 79,000 records as of August 2026, gave every headline number a trail back to the entries behind it.

3. Judgements that need to be consistent

Some tasks have no right answer, only a consistent one. Grading the cosmetic condition of a used device is one. Two experienced people will disagree at the margins and that’s tolerable. The same person grading the same device differently on a different day is not.

A model beats a human here because it doesn’t get tired. Log everything, and test that the same photographs get the same grade every time.

Where it doesn’t pay

When the process is the problem. If three people must approve something and each takes two days, the delay is the approvals, and automating the typing only gets you to the same queue faster.

When the volume isn’t there. A workflow that happens eleven times a month won’t repay a build, a test harness, monitoring and somebody’s continuing attention. Do it by hand and spend the money on the thing that happens eleven thousand times. (Eleven isn’t a magic number. Below a few hundred a month the arithmetic doesn’t work.)

When a report or a rule would do. Either is cheaper than a model, easier to explain to an auditor, and needs no re-testing when a model version changes. When a database query beats a language model has the four questions I ask.

Picking the technique

The reflex that costs most is reaching for the biggest general-purpose model and calling it done.

Prompting an API suits work where the model already knows the domain and you need language handled well. It’s the fastest thing to build and the easiest to get wrong at volume, because you pay per call forever.

Retrieval suits work where the answer lives in your documents. You fetch the relevant history at query time and hand it over, and because it was fetched you can inspect it, cite it and correct it.

Fine-tuning suits work where the domain has structure the model has never seen and can’t infer from a prompt. I fine-tuned a model on domain history for a prediction system, because prompting alone couldn’t get it to a usable answer.

MCP servers suit a problem I now see in every business I look at: staff use AI assistants daily, and the assistant knows nothing about the business it’s being asked about. I built MCP servers over my own business systems so a question about bookings, pipeline or one specific customer is answered from live data, with the permissions that come with it.

And sometimes none of them. A classifier solved twenty years ago, or a database query, will beat prompting, retrieval, fine-tuning and an MCP server on cost, speed and explainability. Get that choice right and the build after it is the part of the job I worry about least.

Right-sizing, and what it does to the bill

A narrow, well-specified task (classify this, extract these fields, decide whether this transcript mentions X) needs the smallest model that clears the bar. At volume, the gap in running cost between that and a frontier model decides whether the workflow pays for itself. So measure the choice, on your own material, because a benchmark published by a model vendor wasn’t run on your documents.

The costs that don’t get quoted

The build is the part that gets quoted. What doesn’t:

• Evaluation. How do you know it still works after the model changes underneath you? If the answer is “someone will notice”, the someone is a customer.
• The unhappy path. What happens when it can’t answer, or answers confidently and wrongly.
• Spend. The per-call bill at full volume, known before month end.

And the data question

For a lot of businesses the data can’t leave. My own business runs the sensitive stages on hardware I own; the case study says why. How to work out which stage that is where you are, and what to take to the person who refused the first proposal, is a separate piece.

What went wrong in the projects I’ve been called into was judgement, and the models had been good enough for years before anybody rang me.