Pilots, probes and experiments

Andi Roberts explaining the difference between pilots, probes and experiments. He then goes on share guidance on how to manage each of them.

Pilots: A pilot is a small, time-bound test to determine whether a product, process, or service can work in practice. It is used when an idea appears promising but remains unproven.

Think of a bank that tries a new mobile feature in one city before a national launch. The trial exposes what could not be seen on paper: user confusion, system issues, or compliance gaps. The value of a pilot lies in its realism. It bridges the space between concept and operation, letting leaders see what truly happens when an idea meets the world. From this, they can refine and strengthen the design before committing at scale.

Probes: A probe begins with curiosity, not certainty. Drawn from complexity thinking, probes are small, safe-to-fail actions that explore what might work when the path ahead is unclear.

A city struggling with congestion might try three different approaches: a cycling subsidy, staggered work hours in one district, and AI-driven traffic lights. None is guaranteed to succeed, and that is the point. Each test offers a glimpse of how the system responds. The power of a probe is its capacity to uncover patterns that analysis alone cannot reveal.

Experiments: An experiment is a structured test designed around a hypothesis. It is used when a leader wants clear evidence about cause and effect.

An online retailer, for example, may compare two website layouts to see which converts more visitors into buyers. Experiments are precise, controlled, and measurable. They do not explore the unknown in the same way that probes do, but they provide reliable evidence where outcomes can be quantified. Their strength lies in giving leaders grounded answers to specific questions.