Why Your AI Project Should Start With A Cost Cap, Not A Model Choice

Ask a team starting an AI project what model they are using and you will get a confident answer, often with benchmarks. Ask what the workflow is allowed to spend per month and the room goes quiet.

That silence is where projects die. Not from bad model choice — from having no answer to “what happens when this costs ten times more than we expected?”

The arithmetic that surprises people

A language-model workflow has a cost that scales with usage in a way ordinary software does not. A web request costs roughly nothing and you never think about it. An agent that reads a document and reasons about it might cost a cent. That is fine at 100 documents a day and alarming at 100,000.

The trap is that the failure mode is not gradual. Costs do not creep when an agent goes wrong — they spike. Common causes:

• Retry loops. The agent fails validation, retries, fails, retries. Each attempt is a full model call. A loop running unattended overnight can burn a month’s budget.
• Growing context. A conversation or document pipeline that appends history means every call is more expensive than the last. Nobody notices until the bill.
• Escalation cascades. A cheap classifier decides to call an expensive reasoner, which decides to call two more. Cost multiplies through the chain.
• Input drift. A new customer sends 40-page PDFs where everyone else sent 2-page ones. Same workflow, 20x the tokens.

None of these are model problems. All of them are missing-limit problems.

Decide the ceiling before the model

Write this down before you write code:

This workflow may spend at most $X per day and $Y per task.
When it hits the limit: <stop / degrade / escalate to human>.

Three numbers and one behaviour. That is the whole design decision, and it is worth more than a week of benchmarking.

The per-task ceiling is your anomaly detector. If a task that normally costs $0.03 suddenly costs $1.20, something is wrong — a retry loop, a runaway context, an input that broke your assumptions. Alert on the ratio, not the total, so growth does not hide the problem.

The daily cap is your circuit breaker. It is not about saving money; it is about bounding the blast radius of a bug. A workflow that can spend without limit is a workflow where one bad deploy is an unbounded incident.

Then pick the model — and pick per task, not per project

Once you have a ceiling, model choice becomes an engineering question instead of an identity question. The useful pattern is routing by difficulty:

• Cheap model for extraction, classification, formatting, and anything with a schema to validate against.
• Expensive model only where genuine reasoning is required, or as a fallback when the cheap model reports low confidence.

This is not a hack to save money. It is the difference between a workflow that costs $0.03 a task and one that costs $0.40 a task for identical output on the 85% of items that were never hard.

The ceiling makes this discipline natural. When you have a budget, you think about where the money goes.

Degrade, don’t die

What should happen at the limit? “Stop and page someone” is usually the wrong answer — you have turned a cost problem into an availability problem.

Better options, in order of preference:

1. Degrade to the cheap model and flag the output for review.
2. Queue the work and process it when the window resets.
3. Escalate to a human with the item and the reason.
4. Stop, but only for the specific workflow that breached — never the whole system.

The right choice depends on whether the work is time-sensitive. A nightly report can wait. A customer-facing response cannot.

The two limits people forget

A global cap as well as per-workflow caps. Ten workflows each within budget can still be a catastrophe in aggregate. Have one number that bounds the account.

An alert before the cap, not at it. Warn at 70%. Hitting the cap is already an incident; the warning is the chance to avoid one.

What this buys you

A workflow with a cost ceiling is one you can safely leave running. That sounds modest. It is actually the entire difference between a demo and a system.

Teams that ship AI successfully are not the ones with the best model. They are the ones whose automations can run unattended without anyone lying awake wondering what the overnight bill looks like.

Start with the cap. The model is the easy part, and it will be a different model in a year anyway.