Nearly every automation project has a business case prepared before approval. Almost none is revisited a year later with actual figures. That review is the cheapest available way to make the next project better, and it takes about a day.
What to measure
Four groups, all recorded from day one rather than reconstructed later.
Output. Units per shift by product, good and rejected separated. Compared against the previous method, measured rather than remembered.
Availability. Actual running hours against intended, stoppages with reasons and durations — including short ones. Short stoppages are usually the largest single loss and they appear in no standard record.
Quality. Pass rate, rework rate, and variation compared with before.
Cost. Actual project cost against budget, and annual running cost including maintenance, consumables, spares and the owner's time.
On the last: running cost is routinely excluded when effectiveness is assessed, and it is a real expense recurring every year. Including it changes the picture materially over a machine's life.
Comparing against the original case
Where the learning is, and it requires honesty.
Rate. What was committed, what is achieved, under which conditions. If short, is the cause the cell design, the input quality, or the operating practice?
Time. Planned duration against actual, and where the slippage occurred.
Cost. Budget against actual spend, and what category the overruns fell into.
Benefits. Did each benefit claimed in the case materialise, and in which account does it appear?
The hardest and most important question. Did the labour saving actually happen? If people were redeployed rather than reduced, the benefit is additional capacity rather than reduced cost — and the case for the next project should be written that way from the start.
Organisations that answer this question honestly write markedly better cases the second time, because they discover which of their assumptions are systematically optimistic.
The results that do not become numbers
Real outcomes that resist quantification and should be recorded separately.
Team capability. How many people can now operate, maintain and configure the system. This is a genuine asset for the next project.
External dependence. How many times the integrator was called this year and for what. The trend matters more than the level.
Attitude on the floor. Whether operators regard the system as a tool or a burden. This determines how the next project is received.
Working conditions. Whether the automated task was a physically demanding one, and what the people who did it are doing now.
Customer effect. Any comment on consistency or capability, and whether any order followed from it.
The first four largely determine whether a second project happens, more than the payback figure does — and they should be reported alongside it rather than left out because they lack units.
Running the review
A structured session a few months after the system has stabilised.
Who attends. Operators, maintenance, the system owner, and whoever approved the investment. Without the first group it becomes a management-only view.
Three questions. What went better than expected, what went worse, what would be done differently.
Write it up. One or two pages, filed, and opened at the start of the next project. Without this step the review is only a conversation.
Share it with the integrator. Honest feedback benefits both sides and it sets the basis for the next engagement.
Keep it separate from accountability. A review where everyone is defending themselves produces nothing. State explicitly that the purpose is to improve the next project.
Include the difficult items. The parts that did not work are where the value is. A review recording only successes has no purpose.
Using it for the next project
The point of the exercise.
Calibrate the estimates. If the first project ran twenty per cent over, apply that factor to the next rather than treating it as an isolated event.
Calibrate the timeline. Particularly the stabilisation period after acceptance, which almost nobody plans for on a first project.
Convert lessons into requirements. If part presentation was the biggest problem, write presentation requirements into the specification rather than leaving the integrator to propose them.
Expand the in-house scope. The second project can be done with less outside help in the areas the team now handles.
Choose the next application from data. After a year of operation you know where the real bottleneck is, which is a far better basis than the initial impression.
An organisation doing this across several projects develops something hard to replicate: knowing precisely what works in its own operation while everyone else is estimating.
Frequently asked questions
Which cost is routinely excluded from effectiveness assessments?
Annual running cost — maintenance, consumables, spares and the owner's time. It recurs every year and including it materially changes the picture over a machine's working life.
What is the hardest question in a post-project review?
Whether the claimed labour saving actually happened. If people were redeployed rather than reduced, the benefit is additional capacity rather than reduced cost, and the next case should be written that way.
Which outcomes determine whether a second project happens?
Team capability, reduced external dependence, and attitude on the floor — more than the payback figure does. They should be reported alongside the numbers rather than omitted for lacking units.
How should the review be framed?
Explicitly as improving the next project rather than as assigning accountability. A review where everyone is defending themselves produces nothing, and the parts that did not work are where the value is.
More in Cost and selection and Humanoids at work.
