Model routing

Model routing is deciding which class of model runs each step of a system, rather than choosing one model and using it everywhere.

Why it matters commercially

Steps within one workflow differ enormously in difficulty. Classifying a document type, extracting a fixed field, and weighing an ambiguous exception are not the same problem, and the price difference between model tiers can be two orders of magnitude per call. A system that uses its most expensive model for every step is usually paying that premium on the majority of calls that never needed it — which is what turns an evaluated pilot into an economics problem at production volume.

How the choice is made

From evaluation, per step, not from a benchmark table. The question is whether a cheaper class produces an acceptable result on that step's actual inputs. Often it does, because the step is narrow. Where it does not, the expensive model is the right answer and its cost is justified by the number of calls it actually receives.

The step with no model

Routing includes the option of routing to nothing. Arithmetic, threshold checks, schema validation, and policy evaluation are more accurate, faster, cheaper, and more auditable as code. Putting a model in that path adds a failure mode and removes an audit trail.

What it is not

Routing is not fallback. A cascade that retries on a larger model after a small one fails is a different mechanism with different economics, and it needs a reliable signal for what counts as failure before it is worth building.