Most robot pilots end with everyone having an opinion and nobody having an answer. The difference between those two outcomes is decided before the pilot starts, in how it is scoped and what gets measured.
Scope it narrowly
The most common pilot mistake is attempting too much.
One task. Not three. A pilot covering several tasks produces ambiguous results because failure in one contaminates judgement of the others.
One location. Not several sites.
One product or item family. Not the full range.
One shift or period. Establish it there before extending.
Why narrow works better. A narrow pilot either succeeds clearly or fails clearly, and both outcomes are useful. A broad pilot usually produces partial results that support any conclusion the reader already held.
Choose the easiest viable task. Not the most valuable one. The pilot's purpose is learning how your organisation works with a robot, and that is learned more cheaply on an easy task.
Organisations that pilot narrowly and then extend consistently outperform those that pilot broadly, because the narrow pilot teaches faster and costs less when it fails.
Write the criteria first
Half a day of work that determines whether the pilot produces a decision.
The primary number. What must be true at the end for this to be worth continuing. Deliveries per shift, questions handled, hours of unattended operation — one number, agreed in advance.
The threshold. Not just what to measure but what value counts as success. This is the part usually skipped and it is the part that makes the pilot decisive.
The measurement method. Who records it, how, how often. Without this the data does not get collected.
Interim expectations. Week one will be poor. Write down what is expected at week one, week four and week eight so a bad start does not end a viable pilot.
The stopping condition. Below what level do you conclude it does not work. Hard to write and it protects against sustaining a failed pilot to avoid admitting it.
Sign it off before starting. With whoever will make the final decision. Criteria agreed after results arrive get negotiated by whoever argues hardest.
Run it long enough
Duration is the second most common mistake after scope.
A week is too short. The first week is dominated by setup problems, novelty and staff unfamiliarity. Nothing measured in week one predicts anything.
Eight to twelve weeks is typical. Enough for the setup problems to be resolved, for staff to form habits, and for a stable measurement period at the end.
Structure it in three phases. Weeks one to two for setup and adjustment, with heavy observation and daily fixes. Weeks three to six for stabilisation, with reducing intervention. Weeks seven onward for measurement, with changes frozen so the data is comparable.
Freeze changes in the final phase. Continuing to adjust during the measurement period makes the numbers meaningless.
Include a difficult period. A busy week, a staff holiday, a product changeover. A pilot that only covers easy conditions has not tested anything.
Measure what will actually decide it
Volume. Tasks completed per shift. The primary number in most pilots.
Interventions. Every time a person had to touch the robot, and why. This is the maintenance burden made visible, and it predicts long-term viability better than volume does.
Availability. Hours running against hours intended, with reasons for the gap.
Quality. Whatever quality means for the task — correct answers, undamaged deliveries, complete inspections.
Staff time. Both saved and spent. Both matter and only the first is usually counted.
Staff opinion. Asked at the start, midpoint and end. Change in opinion over the pilot is a strong signal, and staff usually see problems before the numbers do.
Record daily, not from memory. A single sheet by the machine works better than any system that requires logging in.
Conclude properly
A pilot without a clear conclusion is an expensive way to produce an impression.
Compare against the written criteria. Not against feelings, and not against what the criteria would have been in hindsight.
Three possible outcomes, all legitimate. Proceed and extend. Proceed with changes — usually a different position, task or configuration. Or stop.
Stopping is a success. A pilot that costs a few weeks and prevents an unsuitable purchase has done exactly what a pilot is for. Framing it as a failure discourages anyone from proposing the next one.
Write up what was learned. One or two pages: what worked, what did not, what would be done differently. This is the durable output regardless of the decision.
Extract the data. Conversation logs, task records, the list of things staff reported. Useful independently of the equipment.
Share it internally. Including with the staff who ran it. They participated and they should hear the outcome and the reasoning.
Frequently asked questions
How narrow should a pilot be?
One task, one location, one item family, one period — and the easiest viable task rather than the most valuable one. A narrow pilot succeeds or fails clearly; a broad one produces partial results supporting any conclusion.
What is usually missing from pilot criteria?
The threshold. Teams decide what to measure but not what value counts as success, so the result gets interpreted afterwards by whoever argues most persuasively.
How long should a pilot run?
Eight to twelve weeks, in three phases: setup and adjustment, stabilisation, then a measurement period with changes frozen. It should include at least one difficult period such as a busy week or a changeover.
Which metric best predicts long-term viability?
Interventions — every time a person had to touch the robot and why. This makes the maintenance burden visible, and it predicts whether the robot is still in use in a year better than volume does.
More in Robots in Vietnamese business and Local ecosystem.
