Start with the decision, not the model

Most AI projects that stall were never wrong about the model. They were vague about the decision the model was meant to improve. A team says "we want to use AI for customer support" and six weeks later has a demo that answers questions, a dashboard nobody opens, and no clear answer to the only question that matters: what is different on Monday morning?

The fix is to start one step earlier. Before data, before vendors, before a single notebook, write down the decision. This essay explains how, and works through an example from start to finish.

Name the decision in one sentence

A useful decision statement has a person, a moment and an action. "When a ticket arrives, the support lead decides which queue it goes to." "Before approving a loan, the officer decides whether to ask for more documents." "Each Monday, the store manager decides how much stock to reorder." If you cannot write that sentence, you do not have an AI project yet; you have a curiosity.

Once the sentence exists, three things become obvious that were invisible before:

• Who uses the output. A prediction nobody acts on is decoration.
• When it is needed. A forecast that lands after the decision is made is worthless, however accurate.
• What it costs to be wrong, in each direction. Misrouting a ticket costs minutes. Declining a good borrower costs a customer; approving a bad one costs money. The two errors are rarely equal, and that asymmetry should shape everything that follows.

A worked example: the document request

Consider a lender that makes small business loans. Every application is reviewed by a loan officer, who makes one decision early on: approve the paperwork as complete, or ask the applicant for more documents. The request for more documents is a real cost. It delays the loan by days, frustrates the applicant, and some applicants simply give up. But skipping it when it was needed leads to bad loans or to problems discovered late.

The team's first idea was "use AI to predict loan defaults." That is a large, vague, slow-to-measure goal. Rewritten as a decision, it becomes much smaller and much more useful:

When an application arrives, the loan officer decides whether the documents are sufficient or more are needed.

Measure the decision you already make

Every decision is already being made today, by a person, a rule or a habit. That is your baseline, and it is the number any new system has to beat. In our example the team pulls the last few hundred applications and records, for each: whether more documents were requested, how many days that added, and whether the extra documents actually changed the outcome.

Suppose they find that more documents were requested in a large share of cases, but in most of those cases the extra paperwork changed nothing. That single finding reframes the project. The problem is not predicting defaults. It is that officers request documents defensively, because the cost of missing something is personal to them while the cost of a delay is spread across applicants.

Choose the simplest thing that could change the outcome

With the decision and the baseline written down, the solution space shrinks. In this example the options, from simplest to most complex, are:

1. A checklist written with the most experienced officers, stating which document gaps actually matter for which kind of business.
2. A simple scoring rule that flags applications with known risk patterns for a second look, and lets the rest go through.
3. A model trained on past applications to estimate how likely extra documents are to change the outcome.

The team should try them in that order, for the reasons in Your first model should be boring. The checklist may remove half the unnecessary requests on its own, in a week, with no data science at all. If a model is still worth building afterwards, it now has a precise job, a measured baseline and a clear definition of success: fewer unnecessary requests without more problems discovered late.

Decide how the output reaches the person

A model's output is only useful at the moment of the decision, in the tool where the decision is made. For our loan officer that means a short note on the application screen, such as "documents likely sufficient: similar applications rarely needed more", with the reasons visible. It does not mean a separate dashboard. Every extra click between the prediction and the decision lowers the chance it is used.

It also means deciding in advance who is accountable. The officer still makes the decision; the system informs it. Writing that down early avoids a great deal of confusion later, especially in regulated work.

A one-page decision brief

Everything above fits on one page, written before any technical work starts. It is a close cousin of the one-page product brief:

• Decision: who decides what, and when.
• Baseline: how it is decided today, how long it takes, how often it is wrong.
• Cost of errors: what each kind of mistake costs, and which is worse.
• Options: the simplest things that could improve the outcome, in order.
• Delivery: where and how the output reaches the person at the moment of decision.
• Success: the measurable change you expect, and when you will check it.

Signs you started from the model instead

• The project is named after a technique ("our LLM project") rather than an outcome.
• Nobody can say which person will act differently because of it.
• Success is described as accuracy, with no link to time, money or customers.
• The output lives in a dashboard rather than in the tool where the work happens.
• The current way of making the decision has never been measured.

Any one of these is worth stopping for. Two or more usually mean the project will produce an impressive demo and little else.

Then build

None of this slows a team down. It removes the weeks spent polishing a capability nobody needed, and it gives the eventual model, if one is needed at all, a fair test and a clear purpose. The teams that ship useful AI are rarely the ones with the most sophisticated models. They are the ones who knew, from the first day, which decision they were there to improve, and who measured whether it improved.