Threshold or action matrix: where does a score become an action?

A model returns a probability for each applicant, customer, or machine. On its own, that number approves nobody, calls nobody, and stops nothing. The decision happens one step later, where the institution says what each range of scores should lead to. That step is usually designed in a meeting and changed in a spreadsheet. It deserves the same care as the model.

One threshold is the simplest strategy

The simplest strategy is one cut-off: approve above it, decline below it. Even that is a decision, not a default. The scikit-learn documentation says of its own built-in cut-off rules that they “are most certainly not ideal for most use cases,” and recommends choosing the threshold against a metric the business defines. In credit, the business metric is the institution's risk appetite: how much expected loss it accepts for how much approved volume. That is a policy choice, and the policy owner writes it.

From one threshold to an action matrix

Most real desks need more than approve or decline. They need bands, and for each band an action that also depends on something else: the exposure, the customer's value, the product, the channel. That two-way table is the action matrix.

In an application process, the bands might lead to automatic approval, approval at a lower limit, a credit analyst's review, or a decline. In collections, the same idea runs on the chance of self-cure: a reminder, a call, a payment plan, a pre-legal review. In retention, a churn score crossed with customer value decides who gets a call and who gets no discount at all. Each cell has its action, its rule, and its cap. Every case that falls outside a written cell goes to a named person.

Who owns each cell

The model is ours to build; the matrix is the institution's to write.

1. Credit or risk policy writes the cut-offs and the caps, inside the risk appetite.
2. The approving body (a credit committee, a pricing committee, a risk committee) approves the matrix and every later change to it.
3. Operations owns the review queues and says how many cases a team can actually read in a day.
4. eqlx designs the options, simulates them, and documents the result. We do not set or move a threshold on our own.

Every version of the matrix carries its approval date and approver, so any decision can be traced to the score, the band, the cell, and the matrix version in force that day.

Simulate on history before anything goes live

Before a matrix runs, each option is replayed on the institution's history. For each option the simulation shows the approval volume, the expected bad rate and loss, the size of each review queue, and the swap set: the cases the new strategy treats differently from the current one. The credit team reads a sample of those swapped files. A strategy that looks right in aggregate and wrong on the files it changes is not ready.

The reject-inference caveat

Application history has a blind spot. Crook and Banasik put it plainly: repayment is typically known only for the applicants who were accepted. The behavior of the people who were declined is unknown. A model built only on accepted applicants can carry biased estimates, and, as the same paper notes, cut-offs chosen on an accept-only sample can be wrong for the population of all applicants.

So any simulation below the old cut-off is an extrapolation, and the report says so. The institution has a few honest options: a reject-inference technique with its assumptions stated and tested; outcome data on declined applicants from other sources, where the law allows it; or a small, controlled test below the cut-off, approved by the committee with a written loss cap. None of these removes the uncertainty. Each makes it visible.

Applied in the institution's own systems

The approved matrix runs in the institution's decision engine, limit service, or customer relationship management (CRM) system, not in a separate tool beside them. Each case records its score, model version, band, cell, matrix version, and the action taken, so an auditor can replay any decision. When a case meets no written cell, the engine routes it to a person instead of guessing.

Measured against a control group

A strategy is a claim about what an action will change, and a claim needs a comparison. Where policy allows, a small control group keeps the current treatment: no call, the old limit, the standard offer. The difference in outcomes between the groups is the effect of the action. Where a control group is not acceptable, as for many credit approvals, the comparison runs against the simulated expectation and earlier cohorts, and the report says which comparison was used.

This is outcomes analysis in the sense of the revised US model risk guidance (SR 26-2, April 2026), which “compares model outputs to corresponding real-world outcomes.” It applies to the strategy as much as to the model. The measured result is the evidence for the next threshold change, and the institution approves every change.

A score is an estimate. The bands, the cells, and the actions are the decision, and they belong to the institution that writes them.