Write the stop rule before the pilot starts

A pilot opens into a new segment in February. By August it is still running. Nobody has adopted the motion and nobody has stopped it, because every monthly review reaches the same conclusion: encouraging in places, disappointing in others, too early to say. The spend continues on that sentence.

The results are not the problem. The problem is that no result was ever going to end it.

Ambiguity resolves toward continuing

A test with no decision rule written before exposure has its evidence interpreted afterwards, by people who hold positions on the answer. That is not dishonesty. Mixed results genuinely admit several readings, and the reading that requires nobody to have been wrong is the one that survives the meeting. Continuing is the cheapest outcome socially, so ambiguity converges on it.

Sunk cost takes the blame for this, and it is the wrong diagnosis. A team two quarters into a segment will stop if someone can state plainly that a declared entry condition failed. What the team cannot do is stop on a judgement any colleague can contest with an equally plausible reading of the same numbers. The missing artifact is the rule, not the resolve.

What a decision rule has to contain

A rule capable of ending a pilot names five things, and names them before the first exposure. Written afterwards, each one becomes a negotiation.

1. 01The eligible population, defined by exclusion. Who is in the cohort, and more usefully who is deliberately outside it, so a result cannot be rescued later by widening the boundary.
2. 02The measure, fixed once, with its grain and source named. A conversion rate computed from CRM opportunities and one computed from billed contracts are different measures wearing the same word.
3. 03The decision horizon: the date the question resolves, chosen so the buyer path can complete inside it. A horizon shorter than the buying cycle guarantees an inconclusive answer.
4. 04The threshold, expressed as the range operations will accept rather than a single number, because a single number invites an argument about rounding.
5. 05The action attached to each outcome — continue, reset, stop, stabilize, inconclusive — and specifically what happens in each case.

A rule with no owner is a document

Naming the rule is half the work. The other half is naming who applies it, on what date, with authority to act on the result. Without that, the horizon arrives and the question of whether the threshold was met becomes the same open argument the rule existed to close.

The owner should not be the person whose quarter depends on the answer. That is not a statement about anyone’s integrity. Someone reading evidence that determines their own number is being asked to hold two jobs at once, and the structure should not require them to. Put the decision with whoever owns the operating boundary, and let the team running the motion present the evidence rather than grade it.

Keep the cohort identifiable

A decision rule fails quietly when nobody can reconstruct, six months later, which accounts were actually in the test. Records get merged, owners change, a segment field is overwritten during a cleanup, and the population the rule referred to no longer exists as a queryable set.

Stamp membership at entry and keep it immutable. A cohort identifier written once, on the record, at the moment of exposure, costs nothing at the start and is unrecoverable afterwards. The same applies to the definition itself: store the version of the measure the rule was written against, because a metric quietly redefined in month three turns a clean result into an unfalsifiable one.

Inconclusive needs a rule too

Most decision rules define a pass and a fail. Real exposure produces a third result more often than either: not enough signal to say. If no action is attached to that outcome in advance, it defaults to continuing, which reinstates the failure the rule was written to prevent.

Name the action. Re-run against a larger cohort, reset one named layer, or stop. Any of the three is defensible. Leaving it unstated is what turns a test into a standing budget line.

Reset the smallest defensible layer

When a pilot fails, the instinct is to change the segment, the message, the channel and the offer, then run it again. That produces a second test with no interpretable relationship to the first. Two results exist and neither explains the other.

Change one layer. Hold the rest fixed. The second result then means something relative to the first, and the sequence accumulates into knowledge about the market rather than a list of things that were tried.

Entry is a decision, made once

Market entry is either a test with a declared rule or a commitment with a rationalisation attached. Both are legitimate. A company can decide to enter a segment because it intends to be there in five years, and that decision needs no threshold. What it cannot do is call that a pilot, because the word implies a result that could have gone the other way.

Say which one is being run, and say it in writing before the money moves. Everything downstream — the cohort, the horizon, the reset — follows from that single declaration, and none of it can be recovered once exposure has already happened.